[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:37 dist-test-slave-3kk6 krb5kdc[30279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:37 dist-test-slave-3kk6 krb5kdc[30279](info): set up 2 sockets
Aug 06 06:35:37 dist-test-slave-3kk6 krb5kdc[30279](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 06:35:39.168061 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.373s user 0.002s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:35:39 dist-test-slave-3kk6 krb5kdc[30279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:35:39.301996 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:44655
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:44655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:39.391585 30297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:39.391850 30297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:39.391877 30297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:39.393066 30297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:39.393110 30297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:39.393123 30297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:39.393137 30297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:39.393149 30297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:39.394869 30297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:44655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:44655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:39.395233 30297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:39.395501 30297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:39.402907 30303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:39.402904 30304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:39.402959 30297 server_base.cc:1034] running on GCE node
W20260806 06:35:39.403223 30306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:39.403575 30297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:39.404085 30297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:39.405308 30297 hybrid_clock.cc:648] HybridClock initialized: now 1785998139405295 us; error 60 us; skew 500 ppm
Aug 06 06:35:39 dist-test-slave-3kk6 krb5kdc[30279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998139, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:39.408658 30297 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:35:39.409438 30297 webserver.cc:460] Webserver started at http://127.29.142.190:46389/ using document root <none> and password file <none>
I20260806 06:35:39.409765 30297 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:39.409832 30297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:39.410012 30297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:39.411726 30297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "77a3bf0a3fa64b05aaf64fe949ede6b0"
format_stamp: "Formatted at 2026-08-06 06:35:39 on dist-test-slave-3kk6"
server_key: "6dd80dc128700cd2b09fee5c273d5073"
server_key_iv: "218a1e041a86c296600943c87913647b"
server_key_version: "encryptionkey@0"
I20260806 06:35:39.412431 30297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "77a3bf0a3fa64b05aaf64fe949ede6b0"
format_stamp: "Formatted at 2026-08-06 06:35:39 on dist-test-slave-3kk6"
server_key: "6dd80dc128700cd2b09fee5c273d5073"
server_key_iv: "218a1e041a86c296600943c87913647b"
server_key_version: "encryptionkey@0"
I20260806 06:35:39.414450 30297 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260806 06:35:39.415652 30314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:39.415978 30297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:39.416074 30297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "77a3bf0a3fa64b05aaf64fe949ede6b0"
format_stamp: "Formatted at 2026-08-06 06:35:39 on dist-test-slave-3kk6"
server_key: "6dd80dc128700cd2b09fee5c273d5073"
server_key_iv: "218a1e041a86c296600943c87913647b"
server_key_version: "encryptionkey@0"
I20260806 06:35:39.416186 30297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:39.429476 30297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:39.432030 30297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:39.432248 30297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:39.437520 30297 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:44655
I20260806 06:35:39.437599 30402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:44655 every 8 connection(s)
I20260806 06:35:39.438223 30297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:39.439742 30404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:39.443161 30404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0: Bootstrap starting.
I20260806 06:35:39.444033 30404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:39.444365 30404 log.cc:824] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:39.445272 30404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0: No bootstrap required, opened a new log
I20260806 06:35:39.447201 30404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a3bf0a3fa64b05aaf64fe949ede6b0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 44655 } }
I20260806 06:35:39.447438 30404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:39.447503 30404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77a3bf0a3fa64b05aaf64fe949ede6b0, State: Initialized, Role: FOLLOWER
I20260806 06:35:39.447806 30404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a3bf0a3fa64b05aaf64fe949ede6b0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 44655 } }
I20260806 06:35:39.447985 30404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:39.448055 30404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:39.448113 30404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:39.448437 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 30297
I20260806 06:35:39.448559 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:35:39.448829 30404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a3bf0a3fa64b05aaf64fe949ede6b0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 44655 } }
I20260806 06:35:39.449016 30404 leader_election.cc:302] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77a3bf0a3fa64b05aaf64fe949ede6b0; no voters:
I20260806 06:35:39.449282 30266 external_mini_cluster.cc:1428] Setting key 47f227eb025a26f89ab5c4760d177a59
I20260806 06:35:39.449307 30404 leader_election.cc:288] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:39.449333 30408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:39.449529 30408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [term 1 LEADER]: Becoming Leader. State: Replica: 77a3bf0a3fa64b05aaf64fe949ede6b0, State: Running, Role: LEADER
I20260806 06:35:39.449693 30408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a3bf0a3fa64b05aaf64fe949ede6b0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 44655 } }
I20260806 06:35:39.449822 30404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:39.450222 30408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77a3bf0a3fa64b05aaf64fe949ede6b0. Latest consensus state: current_term: 1 leader_uuid: "77a3bf0a3fa64b05aaf64fe949ede6b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a3bf0a3fa64b05aaf64fe949ede6b0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 44655 } } }
I20260806 06:35:39.450299 30408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:39.451699 30411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77a3bf0a3fa64b05aaf64fe949ede6b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a3bf0a3fa64b05aaf64fe949ede6b0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 44655 } } }
I20260806 06:35:39.452035 30411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0 [sys.catalog]: This master's current role is: LEADER
W20260806 06:35:39.452801 30435 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:35:39.452872 30435 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:35:39.452936 30417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:39.453265 30417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:39.455601 30417 catalog_manager.cc:1347] Generated new cluster ID: 2847f5c3d89a49eab3a2ad2757e19213
I20260806 06:35:39.455662 30417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:35:39 dist-test-slave-3kk6 krb5kdc[30279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:39.468221 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:39.450875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52136 (local address 127.29.142.190:44655)
0806 06:35:39.452164 (+ 1289us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:39.452194 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:39.452237 (+ 43us) server_negotiation.cc:407] Connection header received
0806 06:35:39.454402 (+ 2165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:39.454408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:39.454495 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:39.454651 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:39.456601 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:39.457826 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:39.459156 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:39.459650 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:39.459829 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:39.463687 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:39.463691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:39.463691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:39.463703 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:39.465455 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:39.465980 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:39.465983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:39.465983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:39.466109 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:39.466566 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:39.466571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:39.466572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:39.466898 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:39.467090 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:39.467377 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 06:35:39.467596 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":1180,"thread_start_us":133,"threads_started":1}
I20260806 06:35:39.475031 30417 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:39.475836 30417 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:39.487161 30417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77a3bf0a3fa64b05aaf64fe949ede6b0: Generated new TSK 0
I20260806 06:35:39.487447 30417 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:35:39.517654 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:44655
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:39.612434 30443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:39.612679 30443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:39.612708 30443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:39.614059 30443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:39.614111 30443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:39.614141 30443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:35:39.615841 30443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:44655
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:39.616211 30443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:39.616494 30443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:39.623550 30456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:39.623554 30452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:39.623900 30450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:39.623935 30443 server_base.cc:1034] running on GCE node
I20260806 06:35:39.624154 30443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:39.624434 30443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:39.625582 30443 hybrid_clock.cc:648] HybridClock initialized: now 1785998139625554 us; error 46 us; skew 500 ppm
Aug 06 06:35:39 dist-test-slave-3kk6 krb5kdc[30279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998139, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:39.628522 30443 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:35:39.629083 30443 webserver.cc:460] Webserver started at http://127.29.142.129:40693/ using document root <none> and password file <none>
I20260806 06:35:39.629453 30443 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:39.629520 30443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:39.629694 30443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:39.631078 30443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "f260798636f943c8a4ba5a59d802ff86"
format_stamp: "Formatted at 2026-08-06 06:35:39 on dist-test-slave-3kk6"
server_key: "5b5355cf41597a79e06446371c9b99b1"
server_key_iv: "b0f58032cefb885e2dd80948d984df59"
server_key_version: "encryptionkey@0"
I20260806 06:35:39.631506 30443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "f260798636f943c8a4ba5a59d802ff86"
format_stamp: "Formatted at 2026-08-06 06:35:39 on dist-test-slave-3kk6"
server_key: "5b5355cf41597a79e06446371c9b99b1"
server_key_iv: "b0f58032cefb885e2dd80948d984df59"
server_key_version: "encryptionkey@0"
I20260806 06:35:39.633221 30443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:35:39.634294 30464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:39.634502 30443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:39.634621 30443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "f260798636f943c8a4ba5a59d802ff86"
format_stamp: "Formatted at 2026-08-06 06:35:39 on dist-test-slave-3kk6"
server_key: "5b5355cf41597a79e06446371c9b99b1"
server_key_iv: "b0f58032cefb885e2dd80948d984df59"
server_key_version: "encryptionkey@0"
I20260806 06:35:39.634763 30443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:39.654800 30443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:39.657581 30443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:39.657825 30443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:39.658185 30443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:39.658627 30443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:39.658778 30443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:39.658831 30443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:39.658862 30443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:39.671558 30443 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:37715
I20260806 06:35:39.671684 30627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:37715 every 8 connection(s)
I20260806 06:35:39.672044 30443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:35:39.674335 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 30443
I20260806 06:35:39.674469 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:35:39.674722 30266 external_mini_cluster.cc:1428] Setting key 71797fe56b735053ca4e6c1d36b1b39b
Aug 06 06:35:39 dist-test-slave-3kk6 krb5kdc[30279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998139, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:39.684638 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:39.673221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:46743 (local address 127.29.142.190:44655)
0806 06:35:39.673310 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:39.673312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:39.673499 (+ 187us) server_negotiation.cc:407] Connection header received
0806 06:35:39.674069 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:39.674070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:39.674081 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:39.674144 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:39.675671 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:39.676204 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:39.677839 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:39.678076 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:39.678170 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:39.681082 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:39.681084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:39.681085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:39.681094 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:39.682928 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:39.683522 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:39.683523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:39.683524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:39.683597 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:39.683891 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:39.683892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:39.683892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:39.684054 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:39.684135 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:39.684410 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 06:35:39.684513 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 06:35:39.684676 30632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:39.673166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:44655 (local address 127.29.142.129:46743)
0806 06:35:39.673338 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:39.673385 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:39.673965 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:39.674289 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:39.674294 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:39.675340 (+ 1046us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:39.675590 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:39.675596 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:39.676322 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:39.676324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:39.677730 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:39.677734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:39.678178 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:39.678179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:39.678342 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:39.679169 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:39.679178 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:39.680967 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:39.683030 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:39.683040 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:39.683042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:39.683419 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:39.683680 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:39.683681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:39.683681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:39.683811 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:39.684129 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:39.684134 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:39.684261 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 06:35:39.684348 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:35:39.684493 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260806 06:35:39.685403 30628 heartbeater.cc:347] Connected to a master server at 127.29.142.190:44655
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
I20260806 06:35:39.685535 30628 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:39.685720 30628 heartbeater.cc:511] Master 127.29.142.190:44655 requested a full tablet report, sending...
I20260806 06:35:39.686764 30343 ts_manager.cc:194] Registered new tserver with Master: f260798636f943c8a4ba5a59d802ff86 (127.29.142.129:37715)
I20260806 06:35:39.687508 30343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:46743
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:35:39.706214 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:44655
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:39.797942 30637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:39.798210 30637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:39.798259 30637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:39.799531 30637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:39.799613 30637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:39.799659 30637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:35:39.801208 30637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:44655
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:39.801520 30637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:39.801771 30637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:39.807559 30646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:39.807597 30647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:39.807598 30650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:39.807736 30637 server_base.cc:1034] running on GCE node
I20260806 06:35:39.807992 30637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:39.808298 30637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:39.809486 30637 hybrid_clock.cc:648] HybridClock initialized: now 1785998139809491 us; error 59 us; skew 500 ppm
Aug 06 06:35:39 dist-test-slave-3kk6 krb5kdc[30279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998139, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:39.812388 30637 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:35:39.813062 30637 webserver.cc:460] Webserver started at http://127.29.142.130:37147/ using document root <none> and password file <none>
I20260806 06:35:39.813344 30637 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:39.813426 30637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:39.813613 30637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:39.814960 30637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "32cea332782949768f12d27311e06ec5"
format_stamp: "Formatted at 2026-08-06 06:35:39 on dist-test-slave-3kk6"
server_key: "4209466e23d4547ed7043df91601370d"
server_key_iv: "40440bf9fd8b04e698353a5a52f6c90d"
server_key_version: "encryptionkey@0"
I20260806 06:35:39.815377 30637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "32cea332782949768f12d27311e06ec5"
format_stamp: "Formatted at 2026-08-06 06:35:39 on dist-test-slave-3kk6"
server_key: "4209466e23d4547ed7043df91601370d"
server_key_iv: "40440bf9fd8b04e698353a5a52f6c90d"
server_key_version: "encryptionkey@0"
I20260806 06:35:39.816973 30637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:35:39.817903 30665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:39.818773 30637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:39.818910 30637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "32cea332782949768f12d27311e06ec5"
format_stamp: "Formatted at 2026-08-06 06:35:39 on dist-test-slave-3kk6"
server_key: "4209466e23d4547ed7043df91601370d"
server_key_iv: "40440bf9fd8b04e698353a5a52f6c90d"
server_key_version: "encryptionkey@0"
I20260806 06:35:39.819006 30637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:39.832891 30637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:39.835353 30637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:39.835573 30637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:39.835947 30637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:39.836396 30637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:39.836463 30637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:39.836510 30637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:39.836560 30637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:39.849221 30637 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:33643
I20260806 06:35:39.849300 30837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:33643 every 8 connection(s)
I20260806 06:35:39.849701 30637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:35:39.853391 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 30637
I20260806 06:35:39.853480 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:35:39.853663 30266 external_mini_cluster.cc:1428] Setting key 68236c4409fe7e54fd2e17d33c2b1d27
Aug 06 06:35:39 dist-test-slave-3kk6 krb5kdc[30279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998139, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:39.862275 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:39.850759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:50487 (local address 127.29.142.190:44655)
0806 06:35:39.850847 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:39.850849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:39.851263 (+ 414us) server_negotiation.cc:407] Connection header received
0806 06:35:39.851958 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:39.851959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:39.851970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:39.852037 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:39.853095 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:39.853650 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:39.855423 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:39.855608 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:39.855668 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:39.859111 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:39.859112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:39.859112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:39.859120 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:39.860654 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:39.861209 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:39.861209 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:39.861210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:39.861262 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:39.861593 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:39.861593 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:39.861593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:39.861699 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:39.861746 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:39.862138 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 06:35:39.862206 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 06:35:39.862617 30840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:39.850769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:44655 (local address 127.29.142.130:50487)
0806 06:35:39.851109 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:39.851156 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:39.851853 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:39.852120 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:39.852124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:39.852755 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:39.852998 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:39.853004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:39.853756 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:39.853757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:39.855298 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:39.855310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:39.855715 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:39.855716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:39.855980 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:39.856688 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:39.856698 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:39.858956 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:39.860748 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:39.860757 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:39.860759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:39.861116 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:39.861336 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:39.861336 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:39.861337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:39.861512 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:39.861772 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:39.861780 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:39.861978 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 06:35:39.862106 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 06:35:39.862309 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":217,"threads_started":1}
I20260806 06:35:39.863340 30838 heartbeater.cc:347] Connected to a master server at 127.29.142.190:44655
I20260806 06:35:39.863461 30838 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:39.863876 30838 heartbeater.cc:511] Master 127.29.142.190:44655 requested a full tablet report, sending...
I20260806 06:35:39.864369 30343 ts_manager.cc:194] Registered new tserver with Master: 32cea332782949768f12d27311e06ec5 (127.29.142.130:33643)
I20260806 06:35:39.864789 30343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:50487
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:35:39.886955 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:44655
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:39.982556 30844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:39.982968 30844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:39.983002 30844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:39.984261 30844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:39.984306 30844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:39.984335 30844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:35:39.985898 30844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:44655
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:39.986135 30844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:39.986337 30844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:39.992311 30853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:39.992367 30852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:39.992328 30855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:39.992697 30844 server_base.cc:1034] running on GCE node
I20260806 06:35:39.992977 30844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:39.993268 30844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:39.994455 30844 hybrid_clock.cc:648] HybridClock initialized: now 1785998139994449 us; error 54 us; skew 500 ppm
Aug 06 06:35:39 dist-test-slave-3kk6 krb5kdc[30279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998139, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:39.997332 30844 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:35:39.997967 30844 webserver.cc:460] Webserver started at http://127.29.142.131:42667/ using document root <none> and password file <none>
I20260806 06:35:39.998250 30844 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:39.998330 30844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:39.998521 30844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:39.999800 30844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "f6d979d3e6b147329b2672b3880e2006"
format_stamp: "Formatted at 2026-08-06 06:35:39 on dist-test-slave-3kk6"
server_key: "f2124099a16a122fade7ba0b12a4834f"
server_key_iv: "28dbe7ab889118d7aca1443bec1eac16"
server_key_version: "encryptionkey@0"
I20260806 06:35:40.000259 30844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "f6d979d3e6b147329b2672b3880e2006"
format_stamp: "Formatted at 2026-08-06 06:35:39 on dist-test-slave-3kk6"
server_key: "f2124099a16a122fade7ba0b12a4834f"
server_key_iv: "28dbe7ab889118d7aca1443bec1eac16"
server_key_version: "encryptionkey@0"
I20260806 06:35:40.001933 30844 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:35:40.002916 30869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:40.003211 30844 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:40.003332 30844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "f6d979d3e6b147329b2672b3880e2006"
format_stamp: "Formatted at 2026-08-06 06:35:39 on dist-test-slave-3kk6"
server_key: "f2124099a16a122fade7ba0b12a4834f"
server_key_iv: "28dbe7ab889118d7aca1443bec1eac16"
server_key_version: "encryptionkey@0"
I20260806 06:35:40.003439 30844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:40.024786 30844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:40.027392 30844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:40.027601 30844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:40.027912 30844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:40.028283 30844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:40.028348 30844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:40.028407 30844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:40.028431 30844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:40.038522 30844 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:34751
I20260806 06:35:40.038578 31052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:34751 every 8 connection(s)
I20260806 06:35:40.039033 30844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:35:40.043916 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 30844
I20260806 06:35:40.044039 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:35:40.044207 30266 external_mini_cluster.cc:1428] Setting key d8386ab38b40380587cd9021388ea965
Aug 06 06:35:40 dist-test-slave-3kk6 krb5kdc[30279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998139, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:40.049685 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:40.039995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:55689 (local address 127.29.142.190:44655)
0806 06:35:40.040047 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:40.040050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:40.040285 (+ 235us) server_negotiation.cc:407] Connection header received
0806 06:35:40.040916 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:40.040918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:40.040932 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:40.040979 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:40.042095 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:40.042617 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:40.043756 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:40.043961 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:40.044064 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:40.046608 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:40.046609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:40.046610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:40.046618 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:40.048117 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:40.048569 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:40.048570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:40.048570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:40.048640 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:40.048931 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:40.048932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:40.048933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:40.049061 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:40.049155 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:40.049499 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 06:35:40.049587 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:35:40.049924 31056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:40.039943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:44655 (local address 127.29.142.131:55689)
0806 06:35:40.040131 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:40.040182 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:40.040774 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:40.041070 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:40.041075 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:40.041718 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:40.041964 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:40.041971 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:40.042775 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:40.042776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:40.043627 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:40.043630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:40.044143 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:40.044144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:40.044304 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:40.044986 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:40.044995 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:40.046501 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:40.048227 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:40.048234 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:40.048237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:40.048483 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:40.048726 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:40.048727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:40.048727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:40.048844 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:40.049146 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:40.049154 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:40.049330 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 06:35:40.049462 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 06:35:40.049639 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260806 06:35:40.050519 31053 heartbeater.cc:347] Connected to a master server at 127.29.142.190:44655
I20260806 06:35:40.050647 31053 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:40.050876 31053 heartbeater.cc:511] Master 127.29.142.190:44655 requested a full tablet report, sending...
I20260806 06:35:40.051314 30343 ts_manager.cc:194] Registered new tserver with Master: f6d979d3e6b147329b2672b3880e2006 (127.29.142.131:34751)
I20260806 06:35:40.051733 30343 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:55689
I20260806 06:35:40.056528 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:35:40 dist-test-slave-3kk6 krb5kdc[30279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998140, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:35:40 dist-test-slave-3kk6 krb5kdc[30279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998140, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.142.129@KRBTEST.COM
I20260806 06:35:40.082487 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:40.073990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60096 (local address 127.29.142.129:37715)
0806 06:35:40.074319 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:40.074324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:40.074336 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:35:40.074366 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:40.074368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:40.074384 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:40.074461 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:40.075164 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:40.075945 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:40.076655 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:40.076996 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:40.077053 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:40.079462 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:40.079471 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:40.079471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:40.079478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:40.081280 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:40.081618 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:40.081621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:40.081621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:40.081706 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:40.081880 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:40.081880 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:40.081881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:40.082022 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:40.082203 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:40.082253 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 06:35:40.082349 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":222,"threads_started":1}
W20260806 06:35:40.083094 30507 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60096
Aug 06 06:35:40 dist-test-slave-3kk6 krb5kdc[30279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:35:40 dist-test-slave-3kk6 krb5kdc[30279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.129@KRBTEST.COM
I20260806 06:35:40.113143 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:40.105306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60108 (local address 127.29.142.129:37715)
0806 06:35:40.105403 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:40.105405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:40.105423 (+ 18us) server_negotiation.cc:407] Connection header received
0806 06:35:40.105453 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:40.105453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:40.105495 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:40.105582 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:40.106468 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:40.107422 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:40.108211 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:40.108400 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:40.108454 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:40.110908 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:40.110909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:40.110909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:40.110917 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:40.112166 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:40.112498 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:40.112499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:40.112499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:40.112553 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:40.112741 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:40.112741 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:40.112742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:40.112830 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:40.112931 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:40.113004 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 06:35:40.113071 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 06:35:40.113806 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 30443
I20260806 06:35:40.130002 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 30637
I20260806 06:35:40.140167 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 30844
I20260806 06:35:40.150585 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 30297
2026-08-06T06:35:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2389 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:40 dist-test-slave-3kk6 krb5kdc[31078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:40 dist-test-slave-3kk6 krb5kdc[31078](info): set up 2 sockets
Aug 06 06:35:40 dist-test-slave-3kk6 krb5kdc[31078](info): commencing operation
krb5kdc: starting...
W20260806 06:35:41.374532 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:35:41 dist-test-slave-3kk6 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:35:41.502513 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:45587
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:45587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:41.588366 31098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:41.588595 31098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:41.588618 31098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:41.589854 31098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:41.589907 31098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:41.589922 31098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:41.589936 31098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:41.589947 31098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:41.591589 31098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:45587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:45587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:41.591928 31098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:41.592211 31098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:41.598104 31106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:41.598119 31103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:41.598117 31104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:41.598172 31098 server_base.cc:1034] running on GCE node
I20260806 06:35:41.598567 31098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:41.598954 31098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:41.600117 31098 hybrid_clock.cc:648] HybridClock initialized: now 1785998141600104 us; error 31 us; skew 500 ppm
Aug 06 06:35:41 dist-test-slave-3kk6 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998141, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:41.607542 31098 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:35:41.608197 31098 webserver.cc:460] Webserver started at http://127.29.142.190:44877/ using document root <none> and password file <none>
I20260806 06:35:41.608502 31098 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:41.608558 31098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:41.608757 31098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:41.609927 31098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "2eb7f562a8274d598cf6f98e4b40d910"
format_stamp: "Formatted at 2026-08-06 06:35:41 on dist-test-slave-3kk6"
server_key: "fb1da45dd161030860bc75d4a51e290e"
server_key_iv: "99bb064d096ab46d8fe7cf1216794944"
server_key_version: "encryptionkey@0"
I20260806 06:35:41.610322 31098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "2eb7f562a8274d598cf6f98e4b40d910"
format_stamp: "Formatted at 2026-08-06 06:35:41 on dist-test-slave-3kk6"
server_key: "fb1da45dd161030860bc75d4a51e290e"
server_key_iv: "99bb064d096ab46d8fe7cf1216794944"
server_key_version: "encryptionkey@0"
I20260806 06:35:41.611868 31098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:41.612673 31117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:41.612933 31098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:35:41.613056 31098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "2eb7f562a8274d598cf6f98e4b40d910"
format_stamp: "Formatted at 2026-08-06 06:35:41 on dist-test-slave-3kk6"
server_key: "fb1da45dd161030860bc75d4a51e290e"
server_key_iv: "99bb064d096ab46d8fe7cf1216794944"
server_key_version: "encryptionkey@0"
I20260806 06:35:41.613138 31098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:41.626722 31098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:41.628921 31098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:41.629117 31098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:41.633725 31098 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:45587
I20260806 06:35:41.633785 31196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:45587 every 8 connection(s)
I20260806 06:35:41.634202 31098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:41.635504 31197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:41.637966 31197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910: Bootstrap starting.
I20260806 06:35:41.638701 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 31098
I20260806 06:35:41.638742 31197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:41.638803 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:35:41.638988 30266 external_mini_cluster.cc:1428] Setting key d1378e77fb4b29224a965ffe8f340324
I20260806 06:35:41.639094 31197 log.cc:824] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:41.640096 31197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910: No bootstrap required, opened a new log
I20260806 06:35:41.643085 31197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7f562a8274d598cf6f98e4b40d910" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45587 } }
I20260806 06:35:41.643297 31197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:41.643338 31197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eb7f562a8274d598cf6f98e4b40d910, State: Initialized, Role: FOLLOWER
I20260806 06:35:41.643496 31197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7f562a8274d598cf6f98e4b40d910" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45587 } }
I20260806 06:35:41.643576 31197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:41.643607 31197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:41.643652 31197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:41.644343 31197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7f562a8274d598cf6f98e4b40d910" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45587 } }
I20260806 06:35:41.644510 31197 leader_election.cc:302] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2eb7f562a8274d598cf6f98e4b40d910; no voters:
I20260806 06:35:41.644731 31197 leader_election.cc:288] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:41.644802 31203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:41.645047 31203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [term 1 LEADER]: Becoming Leader. State: Replica: 2eb7f562a8274d598cf6f98e4b40d910, State: Running, Role: LEADER
I20260806 06:35:41.645126 31197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:41.645162 31203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7f562a8274d598cf6f98e4b40d910" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45587 } }
I20260806 06:35:41.645551 31204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2eb7f562a8274d598cf6f98e4b40d910. Latest consensus state: current_term: 1 leader_uuid: "2eb7f562a8274d598cf6f98e4b40d910" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7f562a8274d598cf6f98e4b40d910" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45587 } } }
I20260806 06:35:41.645628 31204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:41.646414 31213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:41.646813 31205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2eb7f562a8274d598cf6f98e4b40d910" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb7f562a8274d598cf6f98e4b40d910" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45587 } } }
I20260806 06:35:41.647019 31213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:41.647100 31205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910 [sys.catalog]: This master's current role is: LEADER
Aug 06 06:35:41 dist-test-slave-3kk6 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:41.649195 31213 catalog_manager.cc:1347] Generated new cluster ID: 04e0ef2834fa441fb0dbf1d9708ab3d5
I20260806 06:35:41.649267 31213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:41.651584 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:41.639639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38146 (local address 127.29.142.190:45587)
0806 06:35:41.639829 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:41.639839 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:41.639871 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:35:41.640437 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:41.640446 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:41.640501 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:41.640630 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:41.641475 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:41.642770 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:41.643902 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:41.644326 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:41.644423 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:41.647817 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:41.647822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:41.647822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:41.647836 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:41.649879 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:41.650299 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:41.650302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:41.650302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:41.650414 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:41.650674 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:41.650675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:41.650676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:41.650936 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:41.651113 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:41.651216 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 06:35:41.651360 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260806 06:35:41.673746 31213 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:41.674324 31213 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:41.678496 31213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2eb7f562a8274d598cf6f98e4b40d910: Generated new TSK 0
I20260806 06:35:41.678750 31213 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:35:41.711980 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:45587
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:41.816907 31230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:41.817238 31230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:41.817268 31230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:41.818508 31230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:41.818552 31230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:41.818580 31230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:35:41.820158 31230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:45587
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:41.820365 31230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:41.820565 31230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:41.825824 31237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:41.825851 31238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:41.825889 31230 server_base.cc:1034] running on GCE node
W20260806 06:35:41.825847 31246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:41.826296 31230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:41.826560 31230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:41.827714 31230 hybrid_clock.cc:648] HybridClock initialized: now 1785998141827689 us; error 42 us; skew 500 ppm
Aug 06 06:35:41 dist-test-slave-3kk6 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998141, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:41.830343 31230 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:35:41.830963 31230 webserver.cc:460] Webserver started at http://127.29.142.129:43657/ using document root <none> and password file <none>
I20260806 06:35:41.831213 31230 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:41.831286 31230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:41.831434 31230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:41.832552 31230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "4116bd3afe2b4026943f56a8d2c2613d"
format_stamp: "Formatted at 2026-08-06 06:35:41 on dist-test-slave-3kk6"
server_key: "b59c28cc5259d7764e34f27e7be8ac93"
server_key_iv: "c3e8bda3d203740f5dc5291338676238"
server_key_version: "encryptionkey@0"
I20260806 06:35:41.832958 31230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "4116bd3afe2b4026943f56a8d2c2613d"
format_stamp: "Formatted at 2026-08-06 06:35:41 on dist-test-slave-3kk6"
server_key: "b59c28cc5259d7764e34f27e7be8ac93"
server_key_iv: "c3e8bda3d203740f5dc5291338676238"
server_key_version: "encryptionkey@0"
I20260806 06:35:41.834375 31230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:35:41.835209 31253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:41.835394 31230 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:41.835503 31230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "4116bd3afe2b4026943f56a8d2c2613d"
format_stamp: "Formatted at 2026-08-06 06:35:41 on dist-test-slave-3kk6"
server_key: "b59c28cc5259d7764e34f27e7be8ac93"
server_key_iv: "c3e8bda3d203740f5dc5291338676238"
server_key_version: "encryptionkey@0"
I20260806 06:35:41.835590 31230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:41.854367 31230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:41.856439 31230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:41.856626 31230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:41.856902 31230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:41.857287 31230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:41.857350 31230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:41.857386 31230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:41.857415 31230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:41.863894 31230 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:33319
I20260806 06:35:41.863947 31408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:33319 every 8 connection(s)
I20260806 06:35:41.864289 31230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:35:41.868822 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 31230
I20260806 06:35:41.868937 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:35:41.869129 30266 external_mini_cluster.cc:1428] Setting key 9fb602e67873fd5c641ed85451c286b9
Aug 06 06:35:41 dist-test-slave-3kk6 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998141, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:41.875402 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:41.865057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:58379 (local address 127.29.142.190:45587)
0806 06:35:41.865159 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:41.865161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:41.865396 (+ 235us) server_negotiation.cc:407] Connection header received
0806 06:35:41.866535 (+ 1139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:41.866537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:41.866549 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:41.866607 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:41.867304 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:41.867830 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:41.869057 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:41.869321 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:41.869391 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:41.872077 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:41.872078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:41.872079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:41.872088 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:41.873708 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:41.874255 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:41.874256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:41.874256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:41.874309 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:41.874712 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:41.874713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:41.874713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:41.874831 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:41.874888 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:41.875254 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 06:35:41.875326 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 06:35:41.875741 31412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:41.865004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:45587 (local address 127.29.142.129:58379)
0806 06:35:41.865239 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:41.865292 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:41.866402 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:41.866710 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:41.866714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:41.866995 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:41.867189 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:41.867193 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:41.867944 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:41.867945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:41.868917 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:41.868921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:41.869432 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:41.869433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:41.869583 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:41.870192 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:41.870204 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:41.871968 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:41.873812 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:41.873820 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:41.873822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:41.874171 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:41.874386 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:41.874387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:41.874387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:41.874582 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:41.874908 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:41.874915 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:41.875096 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 06:35:41.875231 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 06:35:41.875430 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":102,"threads_started":1}
I20260806 06:35:41.876447 31409 heartbeater.cc:347] Connected to a master server at 127.29.142.190:45587
I20260806 06:35:41.876545 31409 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:41.876746 31409 heartbeater.cc:511] Master 127.29.142.190:45587 requested a full tablet report, sending...
I20260806 06:35:41.877332 31142 ts_manager.cc:194] Registered new tserver with Master: 4116bd3afe2b4026943f56a8d2c2613d (127.29.142.129:33319)
I20260806 06:35:41.877979 31142 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:58379
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:35:41.897905 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:45587
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:41.983997 31415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:41.984241 31415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:41.984268 31415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:41.985473 31415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:41.985529 31415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:41.985558 31415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:35:41.987207 31415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:45587
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:41.987504 31415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:41.987783 31415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:41.993155 31427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:41.993137 31429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:41.993139 31423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:41.993556 31415 server_base.cc:1034] running on GCE node
I20260806 06:35:41.993836 31415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:41.994105 31415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:41.995297 31415 hybrid_clock.cc:648] HybridClock initialized: now 1785998141995291 us; error 44 us; skew 500 ppm
Aug 06 06:35:41 dist-test-slave-3kk6 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998141, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:41.998049 31415 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:35:41.998546 31415 webserver.cc:460] Webserver started at http://127.29.142.130:41535/ using document root <none> and password file <none>
I20260806 06:35:41.998836 31415 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:41.998888 31415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:41.998986 31415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:42.000034 31415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "c62939daf67845b2ac37793ca349f89e"
format_stamp: "Formatted at 2026-08-06 06:35:41 on dist-test-slave-3kk6"
server_key: "2a2d1842ee3f779abd88dd179811a81c"
server_key_iv: "60ee4893b5d6c8fc8bc6898b6f784a97"
server_key_version: "encryptionkey@0"
I20260806 06:35:42.000382 31415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "c62939daf67845b2ac37793ca349f89e"
format_stamp: "Formatted at 2026-08-06 06:35:41 on dist-test-slave-3kk6"
server_key: "2a2d1842ee3f779abd88dd179811a81c"
server_key_iv: "60ee4893b5d6c8fc8bc6898b6f784a97"
server_key_version: "encryptionkey@0"
I20260806 06:35:42.001776 31415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:35:42.002498 31437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:42.002740 31415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:42.002866 31415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "c62939daf67845b2ac37793ca349f89e"
format_stamp: "Formatted at 2026-08-06 06:35:41 on dist-test-slave-3kk6"
server_key: "2a2d1842ee3f779abd88dd179811a81c"
server_key_iv: "60ee4893b5d6c8fc8bc6898b6f784a97"
server_key_version: "encryptionkey@0"
I20260806 06:35:42.002949 31415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:42.020474 31415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:42.022572 31415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:42.022797 31415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:42.023016 31415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:42.023371 31415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:42.023408 31415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:42.023470 31415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:42.023501 31415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:42.029623 31415 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:45699
I20260806 06:35:42.029716 31604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:45699 every 8 connection(s)
I20260806 06:35:42.030066 31415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:35:42.033463 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 31415
I20260806 06:35:42.033588 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:35:42.033788 30266 external_mini_cluster.cc:1428] Setting key 00073268c4155db097a2f73db23b8236
Aug 06 06:35:42 dist-test-slave-3kk6 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998141, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:42.041471 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.030934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:49243 (local address 127.29.142.190:45587)
0806 06:35:42.031001 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:42.031003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:42.031313 (+ 310us) server_negotiation.cc:407] Connection header received
0806 06:35:42.032290 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:42.032291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:42.032302 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:42.032357 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:42.032983 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.033478 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.035081 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.035266 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.035326 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.037919 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:42.037920 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:42.037920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:42.037928 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:42.039676 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:42.040292 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:42.040293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:42.040294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:42.040375 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:42.040735 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:42.040736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:42.040736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:42.040856 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:42.040916 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:42.041302 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 06:35:42.041389 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:35:42.041761 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.030892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:45587 (local address 127.29.142.130:49243)
0806 06:35:42.031095 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:42.031175 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:42.032184 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:42.032422 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:42.032426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:42.032698 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:42.032902 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.032907 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.033640 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.033641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.034994 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.034998 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.035358 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.035359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.035505 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.036079 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:42.036090 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:42.037810 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:42.039766 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:42.039776 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:42.039779 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:42.040173 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:42.040470 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:42.040471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:42.040471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:42.040646 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:42.040938 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:42.040945 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:42.041147 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 06:35:42.041284 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 06:35:42.041482 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":103,"threads_started":1}
I20260806 06:35:42.042404 31607 heartbeater.cc:347] Connected to a master server at 127.29.142.190:45587
I20260806 06:35:42.042557 31607 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:42.042804 31607 heartbeater.cc:511] Master 127.29.142.190:45587 requested a full tablet report, sending...
I20260806 06:35:42.043306 31142 ts_manager.cc:194] Registered new tserver with Master: c62939daf67845b2ac37793ca349f89e (127.29.142.130:45699)
I20260806 06:35:42.043717 31142 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:49243
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:35:42.061988 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:45587
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:42.152249 31615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:42.152495 31615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:42.152524 31615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:42.153698 31615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:42.153738 31615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:42.153766 31615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:35:42.155421 31615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:45587
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:42.155637 31615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:42.155836 31615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:35:42.161372 31615 server_base.cc:1034] running on GCE node
W20260806 06:35:42.161336 31620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:42.161306 31622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:42.161314 31625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:42.161744 31615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:42.162039 31615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:42.163226 31615 hybrid_clock.cc:648] HybridClock initialized: now 1785998142163196 us; error 44 us; skew 500 ppm
Aug 06 06:35:42 dist-test-slave-3kk6 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998142, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:42.165978 31615 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:35:42.166496 31615 webserver.cc:460] Webserver started at http://127.29.142.131:35455/ using document root <none> and password file <none>
I20260806 06:35:42.166788 31615 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:42.166843 31615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:42.166945 31615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:42.168068 31615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "a109a0c2d28848e39127827e36fc4b4c"
format_stamp: "Formatted at 2026-08-06 06:35:42 on dist-test-slave-3kk6"
server_key: "f36cc22765ab3786e21b10e2162895e6"
server_key_iv: "acb5e509ae1ec0be9f03e2e5d4c0eb19"
server_key_version: "encryptionkey@0"
I20260806 06:35:42.168395 31615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "a109a0c2d28848e39127827e36fc4b4c"
format_stamp: "Formatted at 2026-08-06 06:35:42 on dist-test-slave-3kk6"
server_key: "f36cc22765ab3786e21b10e2162895e6"
server_key_iv: "acb5e509ae1ec0be9f03e2e5d4c0eb19"
server_key_version: "encryptionkey@0"
I20260806 06:35:42.169739 31615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:42.170462 31638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:42.170734 31615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:35:42.170836 31615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "a109a0c2d28848e39127827e36fc4b4c"
format_stamp: "Formatted at 2026-08-06 06:35:42 on dist-test-slave-3kk6"
server_key: "f36cc22765ab3786e21b10e2162895e6"
server_key_iv: "acb5e509ae1ec0be9f03e2e5d4c0eb19"
server_key_version: "encryptionkey@0"
I20260806 06:35:42.170919 31615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:42.181377 31615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:42.183763 31615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:42.183928 31615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:42.184194 31615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:42.184559 31615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:42.184626 31615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:42.184672 31615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:42.184700 31615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:42.193032 31615 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:46719
I20260806 06:35:42.193104 31812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:46719 every 8 connection(s)
I20260806 06:35:42.193509 31615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:35:42.197858 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 31615
I20260806 06:35:42.198010 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:35:42.198176 30266 external_mini_cluster.cc:1428] Setting key d946e80d4f811dacc8313ac83c02bfcc
Aug 06 06:35:42 dist-test-slave-3kk6 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998142, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:42.204336 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.194587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:54719 (local address 127.29.142.190:45587)
0806 06:35:42.194635 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:42.194637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:42.194929 (+ 292us) server_negotiation.cc:407] Connection header received
0806 06:35:42.195906 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:42.195907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:42.195918 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:42.195986 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:42.196843 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.197350 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.198619 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.199000 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.199090 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.201601 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:42.201602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:42.201602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:42.201610 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:42.202982 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:42.203450 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:42.203450 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:42.203451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:42.203507 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:42.203751 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:42.203752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:42.203752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:42.203862 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:42.203925 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:42.204194 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 06:35:42.204264 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:35:42.204638 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.194552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:45587 (local address 127.29.142.131:54719)
0806 06:35:42.194770 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:42.194824 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:42.195790 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:42.196077 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:42.196081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:42.196413 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:42.196637 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.196642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.197462 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.197463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.198521 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.198525 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.199093 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.199094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.199265 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.199946 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:42.199955 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:42.201498 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:42.203099 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:42.203107 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:42.203110 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:42.203372 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:42.203578 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:42.203579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:42.203579 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:42.203691 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:42.203935 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:42.203940 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:42.204071 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 06:35:42.204161 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 06:35:42.204301 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":132,"threads_started":1}
I20260806 06:35:42.205245 31813 heartbeater.cc:347] Connected to a master server at 127.29.142.190:45587
I20260806 06:35:42.205387 31813 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:42.205667 31813 heartbeater.cc:511] Master 127.29.142.190:45587 requested a full tablet report, sending...
I20260806 06:35:42.206143 31142 ts_manager.cc:194] Registered new tserver with Master: a109a0c2d28848e39127827e36fc4b4c (127.29.142.131:46719)
I20260806 06:35:42.206548 31142 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:54719
I20260806 06:35:42.210289 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:35:42.218868 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.212346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38156 (local address 127.29.142.190:45587)
0806 06:35:42.212420 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:42.212422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:42.212483 (+ 61us) server_negotiation.cc:407] Connection header received
0806 06:35:42.212539 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:42.212540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:42.212549 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:42.212602 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:42.213166 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.213650 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.214348 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.214532 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.214584 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.215857 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:42.215859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:42.215859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:42.215865 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:42.217534 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:42.217956 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:42.217958 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:42.217958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:42.218017 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:42.218236 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:42.218237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:42.218238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:42.218402 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:42.218522 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:42.218626 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:35:42.218741 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:35:42.221841 31142 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:35:42.223858 31142 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:35:42.232177 31833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.227732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35904 (local address 127.29.142.130:45699)
0806 06:35:42.227952 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:42.227957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:42.227969 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:35:42.228021 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:42.228024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:42.228041 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:42.228158 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:42.228488 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.229319 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.230991 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.231698 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.231796 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.231863 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:42.232036 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 06:35:42.232079 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":96,"threads_started":1}
I20260806 06:35:42.232168 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.227434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:45699 (local address 127.0.0.1:35904)
0806 06:35:42.227875 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:42.227886 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:42.227954 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:42.228177 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:42.228181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:42.228183 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:42.228268 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.228287 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.229498 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.229499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.230862 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.230865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.231793 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.231794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.231878 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.231894 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 06:35:42.232012 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 06:35:42.232079 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":28,"spinlock_wait_cycles":896,"thread_start_us":98,"threads_started":1}
I20260806 06:35:42.232309 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.227434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:33319 (local address 127.0.0.1:39234)
0806 06:35:42.227649 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:42.227699 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:42.227786 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:42.228275 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:42.228277 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:42.228279 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:42.228329 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.228331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.229805 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.229807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.230970 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.230973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.232128 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.232128 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.232178 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.232182 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:35:42.232214 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 06:35:42.232248 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":99,"threads_started":1}
I20260806 06:35:42.232343 31834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.227996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48048 (local address 127.29.142.131:46719)
0806 06:35:42.228215 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:42.228219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:42.228230 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:35:42.228268 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:42.228270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:42.228286 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:42.228361 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:42.228597 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.229722 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.230950 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.231758 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.231841 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.231930 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:42.232224 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 06:35:42.232267 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":98,"threads_started":1}
I20260806 06:35:42.232474 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.227689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:46719 (local address 127.0.0.1:48048)
0806 06:35:42.228017 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:42.228028 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:42.228080 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:42.228468 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:42.228469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:42.228472 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:42.228516 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.228517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.229830 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.229831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.230843 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.230846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.231973 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.231992 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.232055 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.232061 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:35:42.232113 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:35:42.232155 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":1,"thread_start_us":137,"threads_started":1}
I20260806 06:35:42.233218 31511 tablet_service.cc:1467] Processing CreateTablet for tablet 996617623fbe4bb2969678a10f87c925 (DEFAULT_TABLE table=test-table [id=7946a7c0bd2142b2ae2dd5d5c2721142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:42.233479 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.227607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.29.142.129:33319)
0806 06:35:42.228080 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:42.228084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:42.228093 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:35:42.228138 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:42.228140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:42.228155 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:42.228273 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:42.228411 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.229638 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.231063 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.232024 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.233241 (+ 1217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.233295 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:42.233319 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 06:35:42.233356 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":140,"threads_started":1}
I20260806 06:35:42.233618 31511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 996617623fbe4bb2969678a10f87c925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:42.234237 31319 tablet_service.cc:1467] Processing CreateTablet for tablet 996617623fbe4bb2969678a10f87c925 (DEFAULT_TABLE table=test-table [id=7946a7c0bd2142b2ae2dd5d5c2721142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:42.234601 31319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 996617623fbe4bb2969678a10f87c925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:42.237087 31835 tablet_bootstrap.cc:492] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e: Bootstrap starting.
I20260806 06:35:42.237582 31712 tablet_service.cc:1467] Processing CreateTablet for tablet 996617623fbe4bb2969678a10f87c925 (DEFAULT_TABLE table=test-table [id=7946a7c0bd2142b2ae2dd5d5c2721142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:42.237879 31836 tablet_bootstrap.cc:492] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d: Bootstrap starting.
I20260806 06:35:42.237928 31835 tablet_bootstrap.cc:654] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:42.237960 31712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 996617623fbe4bb2969678a10f87c925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:42.238619 31836 tablet_bootstrap.cc:654] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:42.238821 31835 log.cc:824] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:42.239362 31836 log.cc:824] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:42.239961 31835 tablet_bootstrap.cc:492] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e: No bootstrap required, opened a new log
I20260806 06:35:42.240072 31835 ts_tablet_manager.cc:1397] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:35:42.240391 31836 tablet_bootstrap.cc:492] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d: No bootstrap required, opened a new log
I20260806 06:35:42.240481 31836 ts_tablet_manager.cc:1397] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 06:35:42.242308 31836 raft_consensus.cc:348] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62939daf67845b2ac37793ca349f89e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 45699 } } peers { permanent_uuid: "a109a0c2d28848e39127827e36fc4b4c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46719 } } peers { permanent_uuid: "4116bd3afe2b4026943f56a8d2c2613d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 33319 } }
I20260806 06:35:42.242551 31836 raft_consensus.cc:374] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:42.242606 31836 raft_consensus.cc:729] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4116bd3afe2b4026943f56a8d2c2613d, State: Initialized, Role: FOLLOWER
I20260806 06:35:42.242717 31835 raft_consensus.cc:348] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62939daf67845b2ac37793ca349f89e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 45699 } } peers { permanent_uuid: "a109a0c2d28848e39127827e36fc4b4c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46719 } } peers { permanent_uuid: "4116bd3afe2b4026943f56a8d2c2613d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 33319 } }
I20260806 06:35:42.242833 31840 tablet_bootstrap.cc:492] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c: Bootstrap starting.
I20260806 06:35:42.242796 31836 consensus_queue.cc:260] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62939daf67845b2ac37793ca349f89e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 45699 } } peers { permanent_uuid: "a109a0c2d28848e39127827e36fc4b4c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46719 } } peers { permanent_uuid: "4116bd3afe2b4026943f56a8d2c2613d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 33319 } }
I20260806 06:35:42.242906 31835 raft_consensus.cc:374] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:42.242955 31835 raft_consensus.cc:729] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62939daf67845b2ac37793ca349f89e, State: Initialized, Role: FOLLOWER
I20260806 06:35:42.243083 31835 consensus_queue.cc:260] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62939daf67845b2ac37793ca349f89e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 45699 } } peers { permanent_uuid: "a109a0c2d28848e39127827e36fc4b4c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46719 } } peers { permanent_uuid: "4116bd3afe2b4026943f56a8d2c2613d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 33319 } }
I20260806 06:35:42.243194 31836 ts_tablet_manager.cc:1428] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 06:35:42.243405 31409 heartbeater.cc:503] Master 127.29.142.190:45587 was elected leader, sending a full tablet report...
I20260806 06:35:42.243427 31607 heartbeater.cc:503] Master 127.29.142.190:45587 was elected leader, sending a full tablet report...
I20260806 06:35:42.243443 31835 ts_tablet_manager.cc:1428] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:35:42.243772 31840 tablet_bootstrap.cc:654] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:42.244468 31840 log.cc:824] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:42.245445 31840 tablet_bootstrap.cc:492] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c: No bootstrap required, opened a new log
I20260806 06:35:42.245502 31840 ts_tablet_manager.cc:1397] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:35:42.247635 31840 raft_consensus.cc:348] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62939daf67845b2ac37793ca349f89e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 45699 } } peers { permanent_uuid: "a109a0c2d28848e39127827e36fc4b4c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46719 } } peers { permanent_uuid: "4116bd3afe2b4026943f56a8d2c2613d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 33319 } }
I20260806 06:35:42.247797 31840 raft_consensus.cc:374] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:42.247819 31840 raft_consensus.cc:729] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a109a0c2d28848e39127827e36fc4b4c, State: Initialized, Role: FOLLOWER
I20260806 06:35:42.247958 31840 consensus_queue.cc:260] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62939daf67845b2ac37793ca349f89e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 45699 } } peers { permanent_uuid: "a109a0c2d28848e39127827e36fc4b4c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46719 } } peers { permanent_uuid: "4116bd3afe2b4026943f56a8d2c2613d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 33319 } }
I20260806 06:35:42.248276 31840 ts_tablet_manager.cc:1428] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:35:42.248345 31813 heartbeater.cc:503] Master 127.29.142.190:45587 was elected leader, sending a full tablet report...
W20260806 06:35:42.280717 31609 tablet.cc:2369] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:35:42.349110 31846 raft_consensus.cc:482] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:35:42.349257 31846 raft_consensus.cc:504] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62939daf67845b2ac37793ca349f89e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 45699 } } peers { permanent_uuid: "a109a0c2d28848e39127827e36fc4b4c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46719 } } peers { permanent_uuid: "4116bd3afe2b4026943f56a8d2c2613d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 33319 } }
I20260806 06:35:42.349859 31846 leader_election.cc:288] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c62939daf67845b2ac37793ca349f89e (127.29.142.130:45699), 4116bd3afe2b4026943f56a8d2c2613d (127.29.142.129:33319)
I20260806 06:35:42.353295 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.350125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:33319 (local address 127.29.142.131:54641)
0806 06:35:42.350193 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:42.350211 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:42.350262 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:42.350517 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:42.350519 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:42.350521 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:42.350604 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.350606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.351434 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.351435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.352302 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.352305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.352960 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.352962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.353088 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.353092 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:35:42.353158 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:35:42.353201 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":19}
I20260806 06:35:42.353303 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.350261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:54641 (local address 127.29.142.129:33319)
0806 06:35:42.350368 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:42.350370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:42.350386 (+ 16us) server_negotiation.cc:407] Connection header received
0806 06:35:42.350419 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:42.350420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:42.350431 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:42.350493 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:42.350741 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.351347 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.352410 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.352877 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.352935 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.352943 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:42.353180 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 06:35:42.353209 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 06:35:42.353626 31833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.350224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:33997 (local address 127.29.142.130:45699)
0806 06:35:42.350291 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:42.350293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:42.350703 (+ 410us) server_negotiation.cc:407] Connection header received
0806 06:35:42.350768 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:42.350769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:42.350785 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:42.350833 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:42.351016 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.351519 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.352681 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.353187 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.353255 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.353262 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:42.353509 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 06:35:42.353554 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:35:42.353873 31847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.350125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:45699 (local address 127.29.142.131:33997)
0806 06:35:42.350599 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:42.350609 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:42.350700 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:42.350864 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:42.350865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:42.350867 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:42.350938 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.350940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.351610 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.351611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.352578 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:42.352581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.353279 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.353280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:42.353408 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.353415 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:35:42.353748 (+ 333us) client_negotiation.cc:240] Negotiation successful
0806 06:35:42.353795 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":221,"thread_start_us":165,"threads_started":1}
I20260806 06:35:42.353850 31351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "996617623fbe4bb2969678a10f87c925" candidate_uuid: "a109a0c2d28848e39127827e36fc4b4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4116bd3afe2b4026943f56a8d2c2613d" is_pre_election: true
I20260806 06:35:42.354100 31351 raft_consensus.cc:2454] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a109a0c2d28848e39127827e36fc4b4c in term 0.
I20260806 06:35:42.354388 31544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "996617623fbe4bb2969678a10f87c925" candidate_uuid: "a109a0c2d28848e39127827e36fc4b4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62939daf67845b2ac37793ca349f89e" is_pre_election: true
I20260806 06:35:42.354444 31647 leader_election.cc:302] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4116bd3afe2b4026943f56a8d2c2613d, a109a0c2d28848e39127827e36fc4b4c; no voters:
I20260806 06:35:42.354553 31544 raft_consensus.cc:2454] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a109a0c2d28848e39127827e36fc4b4c in term 0.
I20260806 06:35:42.354696 31846 raft_consensus.cc:2781] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:35:42.354785 31846 raft_consensus.cc:482] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:35:42.354825 31846 raft_consensus.cc:3037] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:42.355669 31846 raft_consensus.cc:504] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62939daf67845b2ac37793ca349f89e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 45699 } } peers { permanent_uuid: "a109a0c2d28848e39127827e36fc4b4c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46719 } } peers { permanent_uuid: "4116bd3afe2b4026943f56a8d2c2613d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 33319 } }
I20260806 06:35:42.355839 31846 leader_election.cc:288] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [CANDIDATE]: Term 1 election: Requested vote from peers c62939daf67845b2ac37793ca349f89e (127.29.142.130:45699), 4116bd3afe2b4026943f56a8d2c2613d (127.29.142.129:33319)
I20260806 06:35:42.355995 31544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "996617623fbe4bb2969678a10f87c925" candidate_uuid: "a109a0c2d28848e39127827e36fc4b4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c62939daf67845b2ac37793ca349f89e"
I20260806 06:35:42.356076 31544 raft_consensus.cc:3037] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:42.356766 31544 raft_consensus.cc:2454] T 996617623fbe4bb2969678a10f87c925 P c62939daf67845b2ac37793ca349f89e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a109a0c2d28848e39127827e36fc4b4c in term 1.
I20260806 06:35:42.356901 31351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "996617623fbe4bb2969678a10f87c925" candidate_uuid: "a109a0c2d28848e39127827e36fc4b4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4116bd3afe2b4026943f56a8d2c2613d"
I20260806 06:35:42.356933 31641 leader_election.cc:302] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a109a0c2d28848e39127827e36fc4b4c, c62939daf67845b2ac37793ca349f89e; no voters:
I20260806 06:35:42.356982 31351 raft_consensus.cc:3037] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:42.357074 31846 raft_consensus.cc:2781] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:42.357300 31846 raft_consensus.cc:686] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [term 1 LEADER]: Becoming Leader. State: Replica: a109a0c2d28848e39127827e36fc4b4c, State: Running, Role: LEADER
I20260806 06:35:42.357399 31846 consensus_queue.cc:237] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62939daf67845b2ac37793ca349f89e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 45699 } } peers { permanent_uuid: "a109a0c2d28848e39127827e36fc4b4c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46719 } } peers { permanent_uuid: "4116bd3afe2b4026943f56a8d2c2613d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 33319 } }
I20260806 06:35:42.358031 31351 raft_consensus.cc:2454] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a109a0c2d28848e39127827e36fc4b4c in term 1.
I20260806 06:35:42.358732 31140 catalog_manager.cc:5495] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c reported cstate change: term changed from 0 to 1, leader changed from <none> to a109a0c2d28848e39127827e36fc4b4c (127.29.142.131). New cstate: current_term: 1 leader_uuid: "a109a0c2d28848e39127827e36fc4b4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62939daf67845b2ac37793ca349f89e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 45699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a109a0c2d28848e39127827e36fc4b4c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4116bd3afe2b4026943f56a8d2c2613d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 33319 } health_report { overall_health: UNKNOWN } } }
W20260806 06:35:42.365010 31410 tablet.cc:2369] T 996617623fbe4bb2969678a10f87c925 P 4116bd3afe2b4026943f56a8d2c2613d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 06:35:42 dist-test-slave-3kk6 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.129@KRBTEST.COM
I20260806 06:35:42.382918 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.375197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39250 (local address 127.29.142.129:33319)
0806 06:35:42.375269 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:42.375271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:42.375328 (+ 57us) server_negotiation.cc:407] Connection header received
0806 06:35:42.375384 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:42.375385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:42.375396 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:42.375460 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:42.376109 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.376601 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.377337 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.377522 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.377578 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.380112 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:42.380123 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:42.380123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:42.380129 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:42.381764 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:42.382112 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:42.382115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:42.382115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:42.382185 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:42.382364 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:42.382365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:42.382365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:42.382506 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:42.382611 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:42.382771 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 06:35:42.382845 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 06:35:42 dist-test-slave-3kk6 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998142, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:35:42 dist-test-slave-3kk6 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998142, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.142.129@KRBTEST.COM
I20260806 06:35:42.409001 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.401097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.29.142.129:33319)
0806 06:35:42.401223 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:42.401225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:42.401312 (+ 87us) server_negotiation.cc:407] Connection header received
0806 06:35:42.401425 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:42.401426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:42.401443 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:42.401516 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:42.402165 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.402721 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.403494 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.403696 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.403779 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.406376 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:42.406378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:42.406378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:42.406387 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:42.407682 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:42.408066 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:42.408067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:42.408068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:42.408131 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:42.408363 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:42.408363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:42.408364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:42.408469 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:42.408583 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:42.408795 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 06:35:42.408890 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
Aug 06 06:35:42 dist-test-slave-3kk6 krb5kdc[31078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:35:42 dist-test-slave-3kk6 krb5kdc[31078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.129@KRBTEST.COM
I20260806 06:35:42.434196 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:42.426493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39264 (local address 127.29.142.129:33319)
0806 06:35:42.426567 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:42.426570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:42.426723 (+ 153us) server_negotiation.cc:407] Connection header received
0806 06:35:42.426825 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:42.426826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:42.426843 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:42.426918 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:42.427541 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.428028 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.428816 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:42.429006 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:42.429088 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:42.431504 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:42.431505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:42.431506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:42.431514 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:42.433085 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:42.433471 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:42.433472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:42.433472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:42.433529 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:42.433746 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:42.433747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:42.433747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:42.433843 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:42.433927 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:42.434031 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:35:42.434093 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 06:35:42.435135 31285 tablet_service.cc:2908] Rejecting scan request for tablet 996617623fbe4bb2969678a10f87c925: Uninitialized: clean time has not yet been initialized
I20260806 06:35:42.436383 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 31230
W20260806 06:35:42.444320 31814 tablet.cc:2369] T 996617623fbe4bb2969678a10f87c925 P a109a0c2d28848e39127827e36fc4b4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:35:42.447618 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 31415
I20260806 06:35:42.461884 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 31615
I20260806 06:35:42.471398 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 31098
2026-08-06T06:35:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2321 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:42 dist-test-slave-3kk6 krb5kdc[31867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:42 dist-test-slave-3kk6 krb5kdc[31867](info): set up 2 sockets
Aug 06 06:35:42 dist-test-slave-3kk6 krb5kdc[31867](info): commencing operation
krb5kdc: starting...
W20260806 06:35:43.701383 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:35:43 dist-test-slave-3kk6 krb5kdc[31867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:35:43.835224 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:42391
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:42391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:43.922847 31889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:43.923091 31889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:43.923115 31889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:43.924240 31889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:43.924283 31889 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:35:43.924297 31889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:43.924310 31889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:43.924324 31889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:43.924335 31889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:43.925804 31889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:42391
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:42391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:43.926021 31889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:43.926211 31889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:43.931475 31898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:43.931506 31896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:43.931522 31889 server_base.cc:1034] running on GCE node
W20260806 06:35:43.931479 31901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:43.932200 31889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:43.932507 31889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:43.933684 31889 hybrid_clock.cc:648] HybridClock initialized: now 1785998143933659 us; error 46 us; skew 500 ppm
Aug 06 06:35:43 dist-test-slave-3kk6 krb5kdc[31867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998143, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:43.936956 31889 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:35:43.937608 31889 webserver.cc:460] Webserver started at http://127.29.142.190:36219/ using document root <none> and password file <none>
I20260806 06:35:43.937856 31889 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:43.937901 31889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:43.938000 31889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:43.939175 31889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "86a4168c19b8472dadee7ff4ce3cc0da"
format_stamp: "Formatted at 2026-08-06 06:35:43 on dist-test-slave-3kk6"
server_key: "585e7ebe5ce7387d2bc4f1148cf75c97"
server_key_iv: "02688da1ddf7b3f90368e95e1b5fd442"
server_key_version: "encryptionkey@0"
I20260806 06:35:43.939494 31889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "86a4168c19b8472dadee7ff4ce3cc0da"
format_stamp: "Formatted at 2026-08-06 06:35:43 on dist-test-slave-3kk6"
server_key: "585e7ebe5ce7387d2bc4f1148cf75c97"
server_key_iv: "02688da1ddf7b3f90368e95e1b5fd442"
server_key_version: "encryptionkey@0"
I20260806 06:35:43.940975 31889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:35:43.941861 31909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:43.942144 31889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:43.942269 31889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "86a4168c19b8472dadee7ff4ce3cc0da"
format_stamp: "Formatted at 2026-08-06 06:35:43 on dist-test-slave-3kk6"
server_key: "585e7ebe5ce7387d2bc4f1148cf75c97"
server_key_iv: "02688da1ddf7b3f90368e95e1b5fd442"
server_key_version: "encryptionkey@0"
I20260806 06:35:43.942342 31889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:43.951726 31889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:43.954048 31889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:43.954185 31889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:43.961949 31889 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:42391
I20260806 06:35:43.962050 32006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:42391 every 8 connection(s)
I20260806 06:35:43.962436 31889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:43.963446 32008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:43.967044 32008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da: Bootstrap starting.
I20260806 06:35:43.967999 32008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:43.968400 32008 log.cc:824] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:43.969349 32008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da: No bootstrap required, opened a new log
I20260806 06:35:43.971510 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 31889
I20260806 06:35:43.971642 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:35:43.971839 30266 external_mini_cluster.cc:1428] Setting key 7274549476cd125701eedb3ea6dd76bd
I20260806 06:35:43.972729 32008 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a4168c19b8472dadee7ff4ce3cc0da" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 42391 } }
I20260806 06:35:43.972921 32008 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:43.972975 32008 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a4168c19b8472dadee7ff4ce3cc0da, State: Initialized, Role: FOLLOWER
I20260806 06:35:43.973112 32008 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a4168c19b8472dadee7ff4ce3cc0da" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 42391 } }
I20260806 06:35:43.973197 32008 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:43.973233 32008 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:43.973285 32008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:43.973945 32008 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a4168c19b8472dadee7ff4ce3cc0da" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 42391 } }
I20260806 06:35:43.974066 32008 leader_election.cc:302] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86a4168c19b8472dadee7ff4ce3cc0da; no voters:
I20260806 06:35:43.974232 32008 leader_election.cc:288] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:43.974310 32015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:43.974471 32015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [term 1 LEADER]: Becoming Leader. State: Replica: 86a4168c19b8472dadee7ff4ce3cc0da, State: Running, Role: LEADER
I20260806 06:35:43.974500 32008 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:43.974620 32015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a4168c19b8472dadee7ff4ce3cc0da" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 42391 } }
I20260806 06:35:43.975039 32015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86a4168c19b8472dadee7ff4ce3cc0da. Latest consensus state: current_term: 1 leader_uuid: "86a4168c19b8472dadee7ff4ce3cc0da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a4168c19b8472dadee7ff4ce3cc0da" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 42391 } } }
I20260806 06:35:43.975282 32015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:43.975208 32017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86a4168c19b8472dadee7ff4ce3cc0da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a4168c19b8472dadee7ff4ce3cc0da" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 42391 } } }
I20260806 06:35:43.975752 32017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:43.975890 32026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:43.976425 32026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:43.979130 32026 catalog_manager.cc:1347] Generated new cluster ID: 11dbe1b2904f4e40ad24b71751919192
I20260806 06:35:43.979187 32026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:35:43 dist-test-slave-3kk6 krb5kdc[31867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:43.983875 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:43.972502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60026 (local address 127.29.142.190:42391)
0806 06:35:43.972679 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:43.972689 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:43.972727 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:35:43.972836 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:43.972842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:43.972887 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:43.973045 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:43.973866 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:43.975381 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:43.976570 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:43.977014 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:43.977168 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:43.979950 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:43.979954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:43.979954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:43.979966 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:43.982063 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:43.982538 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:43.982542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:43.982542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:43.982702 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:43.982920 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:43.982920 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:43.982921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:43.983235 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:43.983387 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:43.983532 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 06:35:43.983672 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260806 06:35:44.009581 32026 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:44.010396 32026 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:44.013083 32026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da: Generated new TSK 0
I20260806 06:35:44.013304 32026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:35:44.052007 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:42391
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:44.143111 32042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:44.143412 32042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:44.143462 32042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:44.144704 32042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:44.144799 32042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:44.144834 32042 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:35:44.144898 32042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:35:44.146586 32042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.142.190:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:44.146925 32042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:44.147200 32042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:44.153000 32049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:44.153031 32050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:44.153007 32053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:44.153275 32042 server_base.cc:1034] running on GCE node
I20260806 06:35:44.153550 32042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:44.153846 32042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:44.155036 32042 hybrid_clock.cc:648] HybridClock initialized: now 1785998144155023 us; error 52 us; skew 500 ppm
Aug 06 06:35:44 dist-test-slave-3kk6 krb5kdc[31867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:44.157858 32042 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:35:44.158447 32042 webserver.cc:460] Webserver started at http://127.29.142.129:36721/ using document root <none> and password file <none>
I20260806 06:35:44.158779 32042 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:44.158861 32042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:44.159046 32042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:44.160223 32042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "8c96a2c142bf4a5bb31f9509e274400d"
format_stamp: "Formatted at 2026-08-06 06:35:44 on dist-test-slave-3kk6"
server_key: "ac6762396c78243b5af8dd9732aba84b"
server_key_iv: "fee9e5feb96ddd64c6d6f5eaf0fd6fae"
server_key_version: "encryptionkey@0"
I20260806 06:35:44.160679 32042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "8c96a2c142bf4a5bb31f9509e274400d"
format_stamp: "Formatted at 2026-08-06 06:35:44 on dist-test-slave-3kk6"
server_key: "ac6762396c78243b5af8dd9732aba84b"
server_key_iv: "fee9e5feb96ddd64c6d6f5eaf0fd6fae"
server_key_version: "encryptionkey@0"
I20260806 06:35:44.162168 32042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:44.163072 32063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:44.163277 32042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:35:44.163393 32042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "8c96a2c142bf4a5bb31f9509e274400d"
format_stamp: "Formatted at 2026-08-06 06:35:44 on dist-test-slave-3kk6"
server_key: "ac6762396c78243b5af8dd9732aba84b"
server_key_iv: "fee9e5feb96ddd64c6d6f5eaf0fd6fae"
server_key_version: "encryptionkey@0"
I20260806 06:35:44.163486 32042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:44.176662 32042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:44.179417 32042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:44.179649 32042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:44.180315 32042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:44.180394 32042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:44.180444 32042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:44.180501 32042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:35:44 dist-test-slave-3kk6 krb5kdc[31867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:44.189147 32042 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:35711
I20260806 06:35:44.189248 32232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:35711 every 8 connection(s)
I20260806 06:35:44.189651 32042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:35:44.192680 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.181440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:60271 (local address 127.29.142.190:42391)
0806 06:35:44.181496 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.181497 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.181806 (+ 309us) server_negotiation.cc:407] Connection header received
0806 06:35:44.182600 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.182602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.182622 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.182758 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:44.183588 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.184123 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.185195 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.185435 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.185527 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.189074 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:44.189076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:44.189076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:44.189085 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:44.191089 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.191550 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.191551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.191551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.191622 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.191925 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.191926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.191926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.192063 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:44.192141 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.192460 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.192579 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:35:44.192843 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.181427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:42391 (local address 127.29.142.129:60271)
0806 06:35:44.181638 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:44.181679 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:44.182444 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:44.182896 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:44.182902 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:44.183245 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:44.183467 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.183472 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.184237 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.184238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.185085 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.185087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.185536 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.185537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.185687 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.186434 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:44.186445 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:44.188943 (+ 2498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:44.191206 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:44.191213 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:44.191216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:44.191457 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:44.191722 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:44.191723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:44.191724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:44.191842 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:44.192145 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:44.192154 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:44.192309 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 06:35:44.192424 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 06:35:44.192600 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":115,"threads_started":1}
I20260806 06:35:44.193440 32234 heartbeater.cc:347] Connected to a master server at 127.29.142.190:42391
I20260806 06:35:44.193567 32234 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:44.193801 32234 heartbeater.cc:511] Master 127.29.142.190:42391 requested a full tablet report, sending...
I20260806 06:35:44.194406 31935 ts_manager.cc:194] Registered new tserver with Master: 8c96a2c142bf4a5bb31f9509e274400d (127.29.142.129:35711)
I20260806 06:35:44.195261 31935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:60271
I20260806 06:35:44.198464 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 32042
I20260806 06:35:44.198606 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:35:44.198840 30266 external_mini_cluster.cc:1428] Setting key 864d481346520e1170d2f7bd18818261
I20260806 06:35:44.200959 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.194845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:54519 (local address 127.29.142.190:42391)
0806 06:35:44.194888 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.194890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.195029 (+ 139us) server_negotiation.cc:407] Connection header received
0806 06:35:44.195107 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.195108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.195119 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.195177 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:44.195575 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.196116 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.196740 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.196933 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.196980 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.197907 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:44.197908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:44.197909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:44.197915 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:44.199687 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.200091 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.200092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.200092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.200155 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.200386 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.200388 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.200388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.200510 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:44.200588 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.200768 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.200853 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:35:44.200960 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.194769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:42391 (local address 127.29.142.129:54519)
0806 06:35:44.194957 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:44.194967 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:44.195047 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:44.195182 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:44.195183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:44.195414 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:44.195488 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.195490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.196186 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.196187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.196663 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.196664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.196991 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.196991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.197047 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.197508 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:44.197513 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:44.197809 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:44.199790 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:44.199792 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:44.199792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:44.199989 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:44.200235 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:44.200236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:44.200236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:44.200303 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:44.200591 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:44.200592 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:44.200645 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 06:35:44.200728 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:35:44.200854 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:35:44.229911 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:42391
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:44.316677 32249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:44.316916 32249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:44.316943 32249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:44.318090 32249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:44.318128 32249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:44.318149 32249 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:35:44.318167 32249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:35:44.319850 32249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.142.190:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:44.320127 32249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:44.320459 32249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:44.326684 32257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:44.326620 32262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:44.326620 32259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:44.327431 32249 server_base.cc:1034] running on GCE node
I20260806 06:35:44.327684 32249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:44.328003 32249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:44.329238 32249 hybrid_clock.cc:648] HybridClock initialized: now 1785998144329214 us; error 38 us; skew 500 ppm
Aug 06 06:35:44 dist-test-slave-3kk6 krb5kdc[31867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:44.332334 32249 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:35:44.332916 32249 webserver.cc:460] Webserver started at http://127.29.142.130:32785/ using document root <none> and password file <none>
I20260806 06:35:44.333209 32249 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:44.333287 32249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:44.333455 32249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:44.334779 32249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "9c4ba2a2341a4302b465a81bca95ec20"
format_stamp: "Formatted at 2026-08-06 06:35:44 on dist-test-slave-3kk6"
server_key: "12b17a64ed0a4f61c1b2e538266d03c6"
server_key_iv: "68e9f37071693a71c24ac8aec41a5bd5"
server_key_version: "encryptionkey@0"
I20260806 06:35:44.335199 32249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "9c4ba2a2341a4302b465a81bca95ec20"
format_stamp: "Formatted at 2026-08-06 06:35:44 on dist-test-slave-3kk6"
server_key: "12b17a64ed0a4f61c1b2e538266d03c6"
server_key_iv: "68e9f37071693a71c24ac8aec41a5bd5"
server_key_version: "encryptionkey@0"
I20260806 06:35:44.336694 32249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:35:44.337666 32270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:44.337913 32249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:44.338021 32249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "9c4ba2a2341a4302b465a81bca95ec20"
format_stamp: "Formatted at 2026-08-06 06:35:44 on dist-test-slave-3kk6"
server_key: "12b17a64ed0a4f61c1b2e538266d03c6"
server_key_iv: "68e9f37071693a71c24ac8aec41a5bd5"
server_key_version: "encryptionkey@0"
I20260806 06:35:44.338138 32249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:44.348403 32249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:44.350610 32249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:44.350870 32249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:44.351572 32249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:44.351640 32249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:44.351716 32249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:44.351747 32249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:35:44 dist-test-slave-3kk6 krb5kdc[31867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:44.359848 32249 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:36105
I20260806 06:35:44.359880 32425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:36105 every 8 connection(s)
I20260806 06:35:44.360240 32249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:35:44.363111 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.352089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:39251 (local address 127.29.142.190:42391)
0806 06:35:44.352176 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.352179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.352369 (+ 190us) server_negotiation.cc:407] Connection header received
0806 06:35:44.352909 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.352910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.352924 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.353021 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:44.353717 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.354582 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.355699 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.356038 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.356135 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.359782 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:44.359783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:44.359784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:44.359792 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:44.361578 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.362034 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.362035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.362035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.362096 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.362386 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.362386 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.362387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.362516 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:44.362601 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.362929 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.363015 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:35:44.363317 32281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.352042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:42391 (local address 127.29.142.130:39251)
0806 06:35:44.352212 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:44.352259 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:44.352795 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:44.353075 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:44.353078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:44.353339 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:44.353603 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.353609 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.354744 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.354746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.355579 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.355581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.356167 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.356168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.356365 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.357002 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:44.357014 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:44.359647 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:44.361697 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:44.361705 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:44.361707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:44.361946 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:44.362182 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:44.362182 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:44.362183 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:44.362302 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:44.362592 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:44.362599 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:44.362785 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 06:35:44.362907 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 06:35:44.363079 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260806 06:35:44.363909 32426 heartbeater.cc:347] Connected to a master server at 127.29.142.190:42391
I20260806 06:35:44.363993 32426 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:44.364188 32426 heartbeater.cc:511] Master 127.29.142.190:42391 requested a full tablet report, sending...
I20260806 06:35:44.364660 31934 ts_manager.cc:194] Registered new tserver with Master: 9c4ba2a2341a4302b465a81bca95ec20 (127.29.142.130:36105)
I20260806 06:35:44.365185 31934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:39251
I20260806 06:35:44.366216 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 32249
I20260806 06:35:44.366324 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:35:44.366544 30266 external_mini_cluster.cc:1428] Setting key 389b504ec720654beb98cf120c4729ec
I20260806 06:35:44.371486 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.365123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:35171 (local address 127.29.142.190:42391)
0806 06:35:44.365163 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.365165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.365268 (+ 103us) server_negotiation.cc:407] Connection header received
0806 06:35:44.365435 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.365435 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.365443 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.365483 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:44.365879 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.366393 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.367407 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.367612 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.367691 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.368688 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:44.368689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:44.368690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:44.368697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:44.370189 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.370573 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.370574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.370574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.370636 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.370907 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.370908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.370908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.371055 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:44.371139 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.371312 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.371391 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:35:44.371500 32434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.365022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:42391 (local address 127.29.142.130:35171)
0806 06:35:44.365186 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:44.365199 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:44.365269 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:44.365513 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:44.365514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:44.365711 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:44.365779 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.365781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.366487 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.366488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.367296 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.367299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.367703 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.367704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.367780 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.368267 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:44.368272 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:44.368570 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:44.370301 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:44.370302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:44.370302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:44.370489 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:44.370751 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:44.370752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:44.370752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:44.370816 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:44.371132 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:44.371133 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:44.371182 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:35:44.371273 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 06:35:44.371393 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:35:44.395454 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:42391
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:44.483387 32438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:44.483630 32438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:44.483657 32438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:44.484890 32438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:44.484944 32438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:44.484966 32438 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:35:44.484985 32438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:35:44.486577 32438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.142.190:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:44.486935 32438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:44.487205 32438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:44.492870 32447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:44.492870 32450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:44.493124 32438 server_base.cc:1034] running on GCE node
W20260806 06:35:44.492900 32448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:44.493506 32438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:44.493786 32438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:44.494968 32438 hybrid_clock.cc:648] HybridClock initialized: now 1785998144494955 us; error 29 us; skew 500 ppm
Aug 06 06:35:44 dist-test-slave-3kk6 krb5kdc[31867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:44.497805 32438 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:35:44.498334 32438 webserver.cc:460] Webserver started at http://127.29.142.131:39603/ using document root <none> and password file <none>
I20260806 06:35:44.498620 32438 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:44.498764 32438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:44.498947 32438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:44.500124 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "14fd1d18cf7b4488815a7e54dacb663f"
format_stamp: "Formatted at 2026-08-06 06:35:44 on dist-test-slave-3kk6"
server_key: "967a55033c911d88aba71e62e29f2069"
server_key_iv: "7c4d219a04682959896a6c1827a28983"
server_key_version: "encryptionkey@0"
I20260806 06:35:44.500573 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "14fd1d18cf7b4488815a7e54dacb663f"
format_stamp: "Formatted at 2026-08-06 06:35:44 on dist-test-slave-3kk6"
server_key: "967a55033c911d88aba71e62e29f2069"
server_key_iv: "7c4d219a04682959896a6c1827a28983"
server_key_version: "encryptionkey@0"
I20260806 06:35:44.502080 32438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:35:44.502995 32457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:44.503191 32438 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:44.503310 32438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "14fd1d18cf7b4488815a7e54dacb663f"
format_stamp: "Formatted at 2026-08-06 06:35:44 on dist-test-slave-3kk6"
server_key: "967a55033c911d88aba71e62e29f2069"
server_key_iv: "7c4d219a04682959896a6c1827a28983"
server_key_version: "encryptionkey@0"
I20260806 06:35:44.503399 32438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:44.517867 32438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:44.520761 32438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:44.520977 32438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:44.521652 32438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:44.521724 32438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:44.521771 32438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:44.521807 32438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:44.528574 32438 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:38391
I20260806 06:35:44.528710 32622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:38391 every 8 connection(s)
I20260806 06:35:44.529040 32438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:35:44 dist-test-slave-3kk6 krb5kdc[31867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998144, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:44.531672 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 32438
I20260806 06:35:44.531752 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:35:44.531942 30266 external_mini_cluster.cc:1428] Setting key bc507f2916bb37a2818d3448c8b50a43
I20260806 06:35:44.533996 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.522270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:34605 (local address 127.29.142.190:42391)
0806 06:35:44.522347 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.522350 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.522604 (+ 254us) server_negotiation.cc:407] Connection header received
0806 06:35:44.523279 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.523281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.523297 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.523370 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:44.524397 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.525384 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.527091 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.527393 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.527487 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.530452 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:44.530453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:44.530454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:44.530462 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:44.532223 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.532930 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.532930 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.532931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.532990 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.533284 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.533285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.533285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.533414 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:44.533499 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.533811 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.533904 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:35:44.534190 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.522178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:42391 (local address 127.29.142.131:34605)
0806 06:35:44.522450 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:44.522505 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:44.523130 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:44.523455 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:44.523460 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:44.523931 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:44.524279 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.524288 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.525487 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.525488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.526986 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.526989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.527496 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.527497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.527666 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.528426 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:44.528439 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:44.530338 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:44.532323 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:44.532331 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:44.532334 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:44.532658 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:44.533073 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:44.533074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:44.533075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:44.533199 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:44.533490 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:44.533497 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:44.533649 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 06:35:44.533766 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 06:35:44.533941 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":97,"threads_started":1}
I20260806 06:35:44.534832 32623 heartbeater.cc:347] Connected to a master server at 127.29.142.190:42391
I20260806 06:35:44.534940 32623 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:44.535183 32623 heartbeater.cc:511] Master 127.29.142.190:42391 requested a full tablet report, sending...
I20260806 06:35:44.535610 31934 ts_manager.cc:194] Registered new tserver with Master: 14fd1d18cf7b4488815a7e54dacb663f (127.29.142.131:38391)
I20260806 06:35:44.536160 31934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:34605
I20260806 06:35:44.541988 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.536122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:36141 (local address 127.29.142.190:42391)
0806 06:35:44.536168 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.536170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.536252 (+ 82us) server_negotiation.cc:407] Connection header received
0806 06:35:44.536334 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.536335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.536345 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.536391 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:44.536749 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.537342 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.538074 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.538285 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.538374 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.539454 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:44.539456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:44.539456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:44.539465 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:44.540736 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.541145 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.541147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.541147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.541212 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.541441 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.541442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.541443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.541558 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:44.541638 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.541807 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.541887 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:35:44.541991 32631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.536045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:42391 (local address 127.29.142.131:36141)
0806 06:35:44.536185 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:44.536196 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:44.536278 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:44.536405 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:44.536406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:44.536604 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:44.536664 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.536666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.537448 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.537449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.537979 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.537981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.538373 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.538374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.538463 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.539031 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:44.539035 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:44.539342 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:44.540846 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:44.540847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:44.540848 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:44.541064 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:44.541298 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:44.541299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:44.541300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:44.541365 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:44.541629 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:44.541630 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:44.541678 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 06:35:44.541764 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:35:44.541890 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260806 06:35:44.544165 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:35:44 dist-test-slave-3kk6 krb5kdc[31867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998144, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:35:44 dist-test-slave-3kk6 krb5kdc[31867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998144, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:44.569733 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.561965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60034 (local address 127.29.142.190:42391)
0806 06:35:44.562040 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.562041 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.562049 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:35:44.562075 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.562076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.562087 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.562132 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:44.562832 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.563332 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.564180 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.564379 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.564439 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.567098 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:44.567100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:44.567100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:44.567129 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:44.568431 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.568841 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.568843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.568843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.568908 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:44.569153 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:44.569154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:44.569154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:44.569279 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:44.569398 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.569531 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.569621 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:35:44.571709 31934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:35:44.573426 31934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:35:44.581300 32652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.576457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:38391 (local address 127.0.0.1:43098)
0806 06:35:44.576677 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:44.576726 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:44.576805 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:44.577572 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:44.577573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:44.577575 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:44.577612 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.577613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.578511 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.578512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.579955 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.579959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.580871 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.580871 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.580987 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.581005 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 06:35:44.581092 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:35:44.581185 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":73,"thread_start_us":120,"threads_started":1}
I20260806 06:35:44.581300 32655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.576426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:35711 (local address 127.0.0.1:58356)
0806 06:35:44.576893 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:44.576906 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:44.576968 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:44.577308 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:44.577309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:44.577311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:44.577381 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.577384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.578883 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.578884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.579855 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.579858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.580869 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.580871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.580997 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.581005 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 06:35:44.581092 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:35:44.581181 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":2,"thread_start_us":68,"threads_started":1}
I20260806 06:35:44.581912 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.576708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.29.142.131:38391)
0806 06:35:44.577392 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.577396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.577406 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:35:44.577446 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.577448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.577463 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.577574 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:44.577694 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.578395 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.580036 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.580764 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.580835 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.580898 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.581544 (+ 646us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.581795 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":92,"threads_started":1}
I20260806 06:35:44.581310 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.576710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.29.142.129:35711)
0806 06:35:44.577087 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.577091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.577102 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:35:44.577140 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.577142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.577175 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.577256 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:44.577477 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.578749 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.579964 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.580725 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.580820 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.580898 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.581130 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.581193 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":105,"threads_started":1}
I20260806 06:35:44.582808 32131 tablet_service.cc:1467] Processing CreateTablet for tablet acf787c4c18f44c2ac2c5c5fbc6972d0 (DEFAULT_TABLE table=test-table [id=dd34e6b559cc428f8d26fb6ca366d1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:44.583238 32131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acf787c4c18f44c2ac2c5c5fbc6972d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:44.583139 32525 tablet_service.cc:1467] Processing CreateTablet for tablet acf787c4c18f44c2ac2c5c5fbc6972d0 (DEFAULT_TABLE table=test-table [id=dd34e6b559cc428f8d26fb6ca366d1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:44.583482 32525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acf787c4c18f44c2ac2c5c5fbc6972d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:44.586376 32661 tablet_bootstrap.cc:492] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f: Bootstrap starting.
I20260806 06:35:44.586728 32664 tablet_bootstrap.cc:492] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d: Bootstrap starting.
I20260806 06:35:44.587230 32651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.576472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:36105 (local address 127.0.0.1:47216)
0806 06:35:44.576904 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:44.576913 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:44.576958 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:44.577255 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:44.577258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:44.577260 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:44.577368 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.577370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.578850 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.578851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.579959 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.579960 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.582239 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.582241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.586938 (+ 4697us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.586948 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 06:35:44.587037 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 06:35:44.587099 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":57,"thread_start_us":107,"threads_started":1}
I20260806 06:35:44.587256 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.576810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.29.142.130:36105)
0806 06:35:44.576988 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.576994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.577005 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:35:44.577052 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.577054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.577071 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.577155 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:44.577478 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.578767 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.580943 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.582063 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.582144 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.582202 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.587070 (+ 4868us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.587143 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260806 06:35:44.587535 32661 tablet_bootstrap.cc:654] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:44.587559 32664 tablet_bootstrap.cc:654] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:44.587940 32664 log.cc:824] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:44.587919 32661 log.cc:824] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:44.588476 32318 tablet_service.cc:1467] Processing CreateTablet for tablet acf787c4c18f44c2ac2c5c5fbc6972d0 (DEFAULT_TABLE table=test-table [id=dd34e6b559cc428f8d26fb6ca366d1fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:44.588881 32661 tablet_bootstrap.cc:492] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f: No bootstrap required, opened a new log
I20260806 06:35:44.588873 32318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acf787c4c18f44c2ac2c5c5fbc6972d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:44.588974 32661 ts_tablet_manager.cc:1397] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:35:44.589049 32664 tablet_bootstrap.cc:492] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d: No bootstrap required, opened a new log
I20260806 06:35:44.589123 32664 ts_tablet_manager.cc:1397] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:35:44.590485 32661 raft_consensus.cc:348] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.590730 32661 raft_consensus.cc:374] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:44.590787 32661 raft_consensus.cc:729] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14fd1d18cf7b4488815a7e54dacb663f, State: Initialized, Role: FOLLOWER
I20260806 06:35:44.590904 32661 consensus_queue.cc:260] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.591197 32661 ts_tablet_manager.cc:1428] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:35:44.591199 32664 raft_consensus.cc:348] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.591344 32623 heartbeater.cc:503] Master 127.29.142.190:42391 was elected leader, sending a full tablet report...
I20260806 06:35:44.591374 32664 raft_consensus.cc:374] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:44.591423 32664 raft_consensus.cc:729] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c96a2c142bf4a5bb31f9509e274400d, State: Initialized, Role: FOLLOWER
I20260806 06:35:44.591533 32664 consensus_queue.cc:260] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.592283 32664 ts_tablet_manager.cc:1428] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:35:44.592425 32234 heartbeater.cc:503] Master 127.29.142.190:42391 was elected leader, sending a full tablet report...
I20260806 06:35:44.592976 32673 tablet_bootstrap.cc:492] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20: Bootstrap starting.
I20260806 06:35:44.593781 32673 tablet_bootstrap.cc:654] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:44.594074 32673 log.cc:824] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:44.595013 32673 tablet_bootstrap.cc:492] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20: No bootstrap required, opened a new log
I20260806 06:35:44.595111 32673 ts_tablet_manager.cc:1397] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:35:44.596870 32673 raft_consensus.cc:348] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.597054 32673 raft_consensus.cc:374] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:44.597106 32673 raft_consensus.cc:729] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c4ba2a2341a4302b465a81bca95ec20, State: Initialized, Role: FOLLOWER
I20260806 06:35:44.597239 32673 consensus_queue.cc:260] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.597544 32673 ts_tablet_manager.cc:1428] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:35:44.597602 32426 heartbeater.cc:503] Master 127.29.142.190:42391 was elected leader, sending a full tablet report...
W20260806 06:35:44.610946 32427 tablet.cc:2369] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:35:44.690588 32235 tablet.cc:2369] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:35:44.780534 32624 tablet.cc:2369] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:35:44.803788 32676 raft_consensus.cc:482] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:35:44.804005 32676 raft_consensus.cc:504] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.804481 32676 leader_election.cc:288] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c96a2c142bf4a5bb31f9509e274400d (127.29.142.129:35711), 14fd1d18cf7b4488815a7e54dacb663f (127.29.142.131:38391)
I20260806 06:35:44.807890 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.804935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:37853 (local address 127.29.142.129:35711)
0806 06:35:44.804988 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.804990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.805053 (+ 63us) server_negotiation.cc:407] Connection header received
0806 06:35:44.805132 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.805133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.805142 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.805195 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:44.805403 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.805870 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.807116 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.807512 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.807611 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.807621 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.807774 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.807814 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:35:44.807915 32677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.804736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:35711 (local address 127.29.142.130:37853)
0806 06:35:44.804997 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:44.805008 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:44.805067 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:44.805226 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:44.805227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:44.805229 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:44.805323 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.805325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.805977 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.805978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.807009 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.807013 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.807610 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.807611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.807685 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.807690 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:35:44.807768 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:35:44.807811 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":32,"thread_start_us":161,"threads_started":1}
I20260806 06:35:44.808568 32160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf787c4c18f44c2ac2c5c5fbc6972d0" candidate_uuid: "9c4ba2a2341a4302b465a81bca95ec20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c96a2c142bf4a5bb31f9509e274400d" is_pre_election: true
I20260806 06:35:44.808725 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.804812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:48997 (local address 127.29.142.131:38391)
0806 06:35:44.804880 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.804882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.804893 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:35:44.804925 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.804927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.804937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.805036 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:44.805221 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.805878 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.807134 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.807575 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.807662 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.807671 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.808611 (+ 940us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.808652 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:35:44.808717 32281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.804702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:38391 (local address 127.29.142.130:48997)
0806 06:35:44.804764 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:44.804781 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:44.804832 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:44.805048 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:44.805050 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:44.805052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:44.805132 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.805134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.805988 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.805990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.807030 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:44.807034 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.807675 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.807676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:44.808512 (+ 836us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.808516 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:35:44.808601 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:35:44.808643 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":51}
I20260806 06:35:44.808764 32160 raft_consensus.cc:2454] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c4ba2a2341a4302b465a81bca95ec20 in term 0.
I20260806 06:35:44.809018 32273 leader_election.cc:302] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c96a2c142bf4a5bb31f9509e274400d, 9c4ba2a2341a4302b465a81bca95ec20; no voters:
I20260806 06:35:44.809058 32559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf787c4c18f44c2ac2c5c5fbc6972d0" candidate_uuid: "9c4ba2a2341a4302b465a81bca95ec20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14fd1d18cf7b4488815a7e54dacb663f" is_pre_election: true
I20260806 06:35:44.809216 32676 raft_consensus.cc:2781] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:35:44.809283 32676 raft_consensus.cc:482] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:35:44.809319 32676 raft_consensus.cc:3037] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:44.809303 32559 raft_consensus.cc:2454] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c4ba2a2341a4302b465a81bca95ec20 in term 0.
I20260806 06:35:44.810196 32676 raft_consensus.cc:504] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.810374 32676 leader_election.cc:288] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [CANDIDATE]: Term 1 election: Requested vote from peers 8c96a2c142bf4a5bb31f9509e274400d (127.29.142.129:35711), 14fd1d18cf7b4488815a7e54dacb663f (127.29.142.131:38391)
I20260806 06:35:44.810544 32160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf787c4c18f44c2ac2c5c5fbc6972d0" candidate_uuid: "9c4ba2a2341a4302b465a81bca95ec20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c96a2c142bf4a5bb31f9509e274400d"
I20260806 06:35:44.810566 32559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acf787c4c18f44c2ac2c5c5fbc6972d0" candidate_uuid: "9c4ba2a2341a4302b465a81bca95ec20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14fd1d18cf7b4488815a7e54dacb663f"
I20260806 06:35:44.810639 32160 raft_consensus.cc:3037] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:44.810705 32559 raft_consensus.cc:3037] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:44.811468 32160 raft_consensus.cc:2454] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c4ba2a2341a4302b465a81bca95ec20 in term 1.
I20260806 06:35:44.811642 32559 raft_consensus.cc:2454] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c4ba2a2341a4302b465a81bca95ec20 in term 1.
I20260806 06:35:44.811677 32273 leader_election.cc:302] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c96a2c142bf4a5bb31f9509e274400d, 9c4ba2a2341a4302b465a81bca95ec20; no voters:
I20260806 06:35:44.811839 32676 raft_consensus.cc:2781] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:44.812018 32676 raft_consensus.cc:686] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 1 LEADER]: Becoming Leader. State: Replica: 9c4ba2a2341a4302b465a81bca95ec20, State: Running, Role: LEADER
I20260806 06:35:44.812108 32676 consensus_queue.cc:237] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.813136 31935 catalog_manager.cc:5495] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c4ba2a2341a4302b465a81bca95ec20 (127.29.142.130). New cstate: current_term: 1 leader_uuid: "9c4ba2a2341a4302b465a81bca95ec20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } health_report { overall_health: UNKNOWN } } }
I20260806 06:35:44.867236 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:44.864347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.29.142.130:36105)
0806 06:35:44.864398 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:44.864400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:44.864447 (+ 47us) server_negotiation.cc:407] Connection header received
0806 06:35:44.864534 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:44.864535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:44.864553 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:44.864643 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:44.864841 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.865346 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.866247 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:44.866470 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:44.866555 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:44.866807 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:44.866936 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:44.867037 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:44.867082 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 06:35:44.867108 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:35:44.869589 32559 raft_consensus.cc:1261] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f [term 1 FOLLOWER]: Refusing update from remote peer 9c4ba2a2341a4302b465a81bca95ec20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:44.869990 32676 consensus_queue.cc:1035] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:44.871346 32160 raft_consensus.cc:1261] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d [term 1 FOLLOWER]: Refusing update from remote peer 9c4ba2a2341a4302b465a81bca95ec20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:44.871680 32678 consensus_queue.cc:1035] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:44.872851 32682 mvcc.cc:205] Tried to move back new op lower bound from 7315448401382629376 to 7315448401151258624. Current Snapshot: MvccSnapshot[applied={T|T < 7315448401382629376}]
I20260806 06:35:44.873658 32683 mvcc.cc:205] Tried to move back new op lower bound from 7315448401382629376 to 7315448401151258624. Current Snapshot: MvccSnapshot[applied={T|T < 7315448401382629376 or (T in {7315448401382629376})}]
I20260806 06:35:44.876684 32686 mvcc.cc:205] Tried to move back new op lower bound from 7315448401382629376 to 7315448401151258624. Current Snapshot: MvccSnapshot[applied={T|T < 7315448401382629376}]
I20260806 06:35:44.878193 31935 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:35:44.878299 31935 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:35:44.879309 31935 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da: Sending DeleteTablet for 3 replicas of tablet acf787c4c18f44c2ac2c5c5fbc6972d0
I20260806 06:35:44.879819 32318 tablet_service.cc:1514] Processing DeleteTablet for tablet acf787c4c18f44c2ac2c5c5fbc6972d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:44 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:47216
I20260806 06:35:44.879835 32525 tablet_service.cc:1514] Processing DeleteTablet for tablet acf787c4c18f44c2ac2c5c5fbc6972d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:44 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:43098
I20260806 06:35:44.880131 32131 tablet_service.cc:1514] Processing DeleteTablet for tablet acf787c4c18f44c2ac2c5c5fbc6972d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:44 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:58356
I20260806 06:35:44.880357 32694 tablet_replica.cc:330] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f: stopping tablet replica
I20260806 06:35:44.880357 32699 tablet_replica.cc:330] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d: stopping tablet replica
I20260806 06:35:44.880514 32694 raft_consensus.cc:2229] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:35:44.880542 32699 raft_consensus.cc:2229] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:35:44.880647 32694 raft_consensus.cc:2258] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:44.880859 32698 tablet_replica.cc:330] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20: stopping tablet replica
I20260806 06:35:44.880852 31935 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 06:35:44.880964 32698 raft_consensus.cc:2229] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 1 LEADER]: Raft consensus shutting down.
W20260806 06:35:44.881019 31935 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:35:44.881070 32698 raft_consensus.cc:2258] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:44.880899 32699 raft_consensus.cc:2258] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:44.881785 32694 ts_tablet_manager.cc:1905] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:44.882459 32131 tablet_service.cc:1467] Processing CreateTablet for tablet d40685d2c6af48aaa63f66d246bfd2ad (DEFAULT_TABLE table=test-table [id=d8e8b52267f3472eb6a8aff8e2d8b3a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:44.882597 32131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d40685d2c6af48aaa63f66d246bfd2ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:44.882613 32318 tablet_service.cc:1467] Processing CreateTablet for tablet d40685d2c6af48aaa63f66d246bfd2ad (DEFAULT_TABLE table=test-table [id=d8e8b52267f3472eb6a8aff8e2d8b3a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:44.882818 32318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d40685d2c6af48aaa63f66d246bfd2ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:44.884094 32694 ts_tablet_manager.cc:1918] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:35:44.884168 32694 log.cc:1196] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/acf787c4c18f44c2ac2c5c5fbc6972d0
I20260806 06:35:44.884464 32694 ts_tablet_manager.cc:1939] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 14fd1d18cf7b4488815a7e54dacb663f: Deleting consensus metadata
I20260806 06:35:44.884681 32664 tablet_bootstrap.cc:492] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d: Bootstrap starting.
I20260806 06:35:44.885134 32664 tablet_bootstrap.cc:654] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:44.885596 32673 tablet_bootstrap.cc:492] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20: Bootstrap starting.
I20260806 06:35:44.885910 32664 tablet_bootstrap.cc:492] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d: No bootstrap required, opened a new log
I20260806 06:35:44.885947 32664 ts_tablet_manager.cc:1397] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:44.886070 32673 tablet_bootstrap.cc:654] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:44.886142 32664 raft_consensus.cc:348] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.886196 32664 raft_consensus.cc:374] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:44.886210 32664 raft_consensus.cc:729] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c96a2c142bf4a5bb31f9509e274400d, State: Initialized, Role: FOLLOWER
I20260806 06:35:44.886287 32664 consensus_queue.cc:260] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.886444 32664 ts_tablet_manager.cc:1428] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:35:44.886847 31910 catalog_manager.cc:4841] TS 14fd1d18cf7b4488815a7e54dacb663f (127.29.142.131:38391): tablet acf787c4c18f44c2ac2c5c5fbc6972d0 (table test-table [id=dd34e6b559cc428f8d26fb6ca366d1fe]) successfully deleted
I20260806 06:35:44.886952 32525 tablet_service.cc:1467] Processing CreateTablet for tablet d40685d2c6af48aaa63f66d246bfd2ad (DEFAULT_TABLE table=test-table [id=d8e8b52267f3472eb6a8aff8e2d8b3a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:44.887073 32525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d40685d2c6af48aaa63f66d246bfd2ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:44.887732 32699 ts_tablet_manager.cc:1905] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:44.888087 32698 ts_tablet_manager.cc:1905] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:44.888335 32661 tablet_bootstrap.cc:492] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f: Bootstrap starting.
I20260806 06:35:44.888796 32661 tablet_bootstrap.cc:654] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:44.889547 32661 tablet_bootstrap.cc:492] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f: No bootstrap required, opened a new log
I20260806 06:35:44.889626 32661 ts_tablet_manager.cc:1397] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:35:44.889863 32661 raft_consensus.cc:348] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.889966 32661 raft_consensus.cc:374] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:44.889997 32661 raft_consensus.cc:729] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14fd1d18cf7b4488815a7e54dacb663f, State: Initialized, Role: FOLLOWER
I20260806 06:35:44.890044 32661 consensus_queue.cc:260] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.890138 32661 ts_tablet_manager.cc:1428] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:44.890201 32698 ts_tablet_manager.cc:1918] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:35:44.890285 32698 log.cc:1196] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/acf787c4c18f44c2ac2c5c5fbc6972d0
I20260806 06:35:44.890564 32698 ts_tablet_manager.cc:1939] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 9c4ba2a2341a4302b465a81bca95ec20: Deleting consensus metadata
I20260806 06:35:44.891036 31916 catalog_manager.cc:4841] TS 9c4ba2a2341a4302b465a81bca95ec20 (127.29.142.130:36105): tablet acf787c4c18f44c2ac2c5c5fbc6972d0 (table test-table [id=dd34e6b559cc428f8d26fb6ca366d1fe]) successfully deleted
I20260806 06:35:44.892943 32699 ts_tablet_manager.cc:1918] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:35:44.893033 32699 log.cc:1196] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/wal/wals/acf787c4c18f44c2ac2c5c5fbc6972d0
I20260806 06:35:44.893321 32699 ts_tablet_manager.cc:1939] T acf787c4c18f44c2ac2c5c5fbc6972d0 P 8c96a2c142bf4a5bb31f9509e274400d: Deleting consensus metadata
I20260806 06:35:44.893859 31911 catalog_manager.cc:4841] TS 8c96a2c142bf4a5bb31f9509e274400d (127.29.142.129:35711): tablet acf787c4c18f44c2ac2c5c5fbc6972d0 (table test-table [id=dd34e6b559cc428f8d26fb6ca366d1fe]) successfully deleted
I20260806 06:35:44.894073 32673 tablet_bootstrap.cc:492] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20: No bootstrap required, opened a new log
I20260806 06:35:44.894134 32673 ts_tablet_manager.cc:1397] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260806 06:35:44.894317 32673 raft_consensus.cc:348] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.894438 32673 raft_consensus.cc:374] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:44.894461 32673 raft_consensus.cc:729] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c4ba2a2341a4302b465a81bca95ec20, State: Initialized, Role: FOLLOWER
I20260806 06:35:44.894531 32673 consensus_queue.cc:260] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:44.894635 32673 ts_tablet_manager.cc:1428] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:45.077714 32672 raft_consensus.cc:482] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:35:45.077844 32672 raft_consensus.cc:504] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:45.078294 32672 leader_election.cc:288] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c4ba2a2341a4302b465a81bca95ec20 (127.29.142.130:36105), 14fd1d18cf7b4488815a7e54dacb663f (127.29.142.131:38391)
I20260806 06:35:45.082003 32704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.078523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:36105 (local address 127.29.142.129:55015)
0806 06:35:45.078851 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:45.078861 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:45.078955 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:45.079117 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:45.079118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:45.079120 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:45.079209 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:45.079211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.079908 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.079909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:45.080928 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:45.080930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.081649 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.081650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:45.081747 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.081750 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:35:45.081834 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:35:45.081881 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":75,"threads_started":1}
I20260806 06:35:45.081964 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.078604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:55015 (local address 127.29.142.130:36105)
0806 06:35:45.078704 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:45.078707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:45.078938 (+ 231us) server_negotiation.cc:407] Connection header received
0806 06:35:45.079030 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:45.079031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:45.079044 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:45.079102 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:45.079291 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.079784 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.081041 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.081438 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.081535 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.081546 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:45.081841 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 06:35:45.081894 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:35:45.082060 32703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.078520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:38391 (local address 127.29.142.129:34641)
0806 06:35:45.078691 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:45.078705 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:45.078786 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:45.079013 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:45.079014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:45.079017 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:45.079174 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:45.079177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.079902 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.079903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:45.080915 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:45.080918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.081543 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.081545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:45.081662 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.081665 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:35:45.081748 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:35:45.081793 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":47,"thread_start_us":255,"threads_started":1}
I20260806 06:35:45.082052 32656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.078678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:34641 (local address 127.29.142.131:38391)
0806 06:35:45.078728 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:45.078730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:45.078766 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:35:45.078910 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:45.078911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:45.078926 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:45.079000 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:45.079273 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.079785 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.081039 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.081427 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.081523 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.081535 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:45.081923 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 06:35:45.081972 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:35:45.082310 32559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d40685d2c6af48aaa63f66d246bfd2ad" candidate_uuid: "8c96a2c142bf4a5bb31f9509e274400d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14fd1d18cf7b4488815a7e54dacb663f" is_pre_election: true
I20260806 06:35:45.082441 32559 raft_consensus.cc:2454] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c96a2c142bf4a5bb31f9509e274400d in term 0.
I20260806 06:35:45.082337 32361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d40685d2c6af48aaa63f66d246bfd2ad" candidate_uuid: "8c96a2c142bf4a5bb31f9509e274400d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c4ba2a2341a4302b465a81bca95ec20" is_pre_election: true
I20260806 06:35:45.082525 32361 raft_consensus.cc:2454] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c96a2c142bf4a5bb31f9509e274400d in term 0.
I20260806 06:35:45.082729 32064 leader_election.cc:302] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14fd1d18cf7b4488815a7e54dacb663f, 8c96a2c142bf4a5bb31f9509e274400d; no voters:
I20260806 06:35:45.082904 32672 raft_consensus.cc:2781] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:35:45.082970 32672 raft_consensus.cc:482] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:35:45.083005 32672 raft_consensus.cc:3037] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:45.083774 32672 raft_consensus.cc:504] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:45.083979 32672 leader_election.cc:288] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [CANDIDATE]: Term 1 election: Requested vote from peers 9c4ba2a2341a4302b465a81bca95ec20 (127.29.142.130:36105), 14fd1d18cf7b4488815a7e54dacb663f (127.29.142.131:38391)
I20260806 06:35:45.084167 32559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d40685d2c6af48aaa63f66d246bfd2ad" candidate_uuid: "8c96a2c142bf4a5bb31f9509e274400d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14fd1d18cf7b4488815a7e54dacb663f"
I20260806 06:35:45.084246 32559 raft_consensus.cc:3037] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:45.085047 32559 raft_consensus.cc:2454] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c96a2c142bf4a5bb31f9509e274400d in term 1.
I20260806 06:35:45.085199 32361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d40685d2c6af48aaa63f66d246bfd2ad" candidate_uuid: "8c96a2c142bf4a5bb31f9509e274400d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c4ba2a2341a4302b465a81bca95ec20"
I20260806 06:35:45.085260 32064 leader_election.cc:302] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14fd1d18cf7b4488815a7e54dacb663f, 8c96a2c142bf4a5bb31f9509e274400d; no voters:
I20260806 06:35:45.085323 32361 raft_consensus.cc:3037] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:45.085369 32672 raft_consensus.cc:2781] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:45.085530 32672 raft_consensus.cc:686] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 1 LEADER]: Becoming Leader. State: Replica: 8c96a2c142bf4a5bb31f9509e274400d, State: Running, Role: LEADER
I20260806 06:35:45.085613 32672 consensus_queue.cc:237] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:45.086369 32361 raft_consensus.cc:2454] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c96a2c142bf4a5bb31f9509e274400d in term 1.
I20260806 06:35:45.086375 31935 catalog_manager.cc:5495] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c96a2c142bf4a5bb31f9509e274400d (127.29.142.129). New cstate: current_term: 1 leader_uuid: "8c96a2c142bf4a5bb31f9509e274400d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } health_report { overall_health: UNKNOWN } } }
Aug 06 06:35:45 dist-test-slave-3kk6 krb5kdc[31867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998143, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:45.143643 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.135753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:42391 (local address 127.0.0.1:60050)
0806 06:35:45.136026 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:45.136040 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:45.136124 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:45.136282 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:45.136282 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:45.136717 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:45.136772 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:45.136775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.137451 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.137452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:45.138008 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:45.138010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.138354 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.138354 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:45.138395 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.139017 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:45.139022 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:45.140601 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:45.142558 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:45.142568 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:45.142570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:45.142869 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:45.143070 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:45.143071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:45.143071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:45.143131 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:45.143363 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:45.143370 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:45.143414 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 06:35:45.143464 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:35:45.143561 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":126,"threads_started":1}
I20260806 06:35:45.143720 32715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.135839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60050 (local address 127.29.142.190:42391)
0806 06:35:45.136039 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:45.136041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:45.136119 (+ 78us) server_negotiation.cc:407] Connection header received
0806 06:35:45.136181 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:45.136182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:45.136192 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:45.136261 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:45.136849 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.137353 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.138090 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.138280 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.138335 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.140726 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:45.140728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:45.140728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:45.140737 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:45.142473 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:45.142942 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:45.142943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:45.142943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:45.143005 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:45.143190 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:45.143190 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:45.143190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:45.143305 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:45.143358 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:45.143546 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 06:35:45.143621 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":151,"threads_started":1}
I20260806 06:35:45.146209 31935 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:60050:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:35:45.146525 31935 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:35:45.148222 32318 tablet_service.cc:1467] Processing CreateTablet for tablet 56ea3b389ace42849ae3f0405b3eb564 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=675cb3771b25451fbda614aece827fa6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:35:45.148247 32525 tablet_service.cc:1467] Processing CreateTablet for tablet 56ea3b389ace42849ae3f0405b3eb564 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=675cb3771b25451fbda614aece827fa6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:35:45.148491 32525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56ea3b389ace42849ae3f0405b3eb564. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:45.148490 32318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56ea3b389ace42849ae3f0405b3eb564. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:45.149905 32673 tablet_bootstrap.cc:492] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20: Bootstrap starting.
I20260806 06:35:45.149940 32661 tablet_bootstrap.cc:492] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f: Bootstrap starting.
I20260806 06:35:45.150022 32131 tablet_service.cc:1467] Processing CreateTablet for tablet 56ea3b389ace42849ae3f0405b3eb564 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=675cb3771b25451fbda614aece827fa6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:35:45.150290 32131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56ea3b389ace42849ae3f0405b3eb564. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:45.150408 32673 tablet_bootstrap.cc:654] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:45.150426 32661 tablet_bootstrap.cc:654] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:45.151117 32673 tablet_bootstrap.cc:492] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20: No bootstrap required, opened a new log
I20260806 06:35:45.151165 32673 ts_tablet_manager.cc:1397] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:35:45.151297 32673 raft_consensus.cc:348] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:45.151384 32673 raft_consensus.cc:374] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:45.151400 32673 raft_consensus.cc:729] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c4ba2a2341a4302b465a81bca95ec20, State: Initialized, Role: FOLLOWER
I20260806 06:35:45.151424 32673 consensus_queue.cc:260] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:45.151546 32673 ts_tablet_manager.cc:1428] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:45.151574 32691 tablet_replica.cc:439] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } } }
I20260806 06:35:45.151683 32691 tablet_replica.cc:442] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:45.152024 32664 tablet_bootstrap.cc:492] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: Bootstrap starting.
I20260806 06:35:45.152865 32661 tablet_bootstrap.cc:492] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f: No bootstrap required, opened a new log
I20260806 06:35:45.152902 32661 ts_tablet_manager.cc:1397] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 06:35:45.152936 32664 tablet_bootstrap.cc:654] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:45.153141 32661 raft_consensus.cc:348] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:45.153195 32661 raft_consensus.cc:374] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:45.153208 32661 raft_consensus.cc:729] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14fd1d18cf7b4488815a7e54dacb663f, State: Initialized, Role: FOLLOWER
I20260806 06:35:45.153280 32661 consensus_queue.cc:260] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:45.153389 32661 ts_tablet_manager.cc:1428] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:45.153554 32668 tablet_replica.cc:439] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } } }
I20260806 06:35:45.153652 32668 tablet_replica.cc:442] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:45.154170 32664 tablet_bootstrap.cc:492] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: No bootstrap required, opened a new log
I20260806 06:35:45.154253 32664 ts_tablet_manager.cc:1397] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:35:45.154527 32664 raft_consensus.cc:348] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:45.154633 32664 raft_consensus.cc:374] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:45.154661 32664 raft_consensus.cc:729] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c96a2c142bf4a5bb31f9509e274400d, State: Initialized, Role: FOLLOWER
I20260806 06:35:45.154736 32664 consensus_queue.cc:260] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:45.154836 32664 ts_tablet_manager.cc:1428] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:35:45.154917 32672 tablet_replica.cc:439] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } } }
I20260806 06:35:45.155022 32672 tablet_replica.cc:442] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:45.168081 32672 raft_consensus.cc:482] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:35:45.168188 32672 raft_consensus.cc:504] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:45.168371 32672 leader_election.cc:288] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c4ba2a2341a4302b465a81bca95ec20 (127.29.142.130:36105), 14fd1d18cf7b4488815a7e54dacb663f (127.29.142.131:38391)
I20260806 06:35:45.168618 32559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56ea3b389ace42849ae3f0405b3eb564" candidate_uuid: "8c96a2c142bf4a5bb31f9509e274400d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14fd1d18cf7b4488815a7e54dacb663f" is_pre_election: true
I20260806 06:35:45.168615 32361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56ea3b389ace42849ae3f0405b3eb564" candidate_uuid: "8c96a2c142bf4a5bb31f9509e274400d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c4ba2a2341a4302b465a81bca95ec20" is_pre_election: true
I20260806 06:35:45.168718 32559 raft_consensus.cc:2454] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c96a2c142bf4a5bb31f9509e274400d in term 0.
I20260806 06:35:45.168741 32361 raft_consensus.cc:2454] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c96a2c142bf4a5bb31f9509e274400d in term 0.
I20260806 06:35:45.168990 32068 leader_election.cc:302] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c96a2c142bf4a5bb31f9509e274400d, 9c4ba2a2341a4302b465a81bca95ec20; no voters:
I20260806 06:35:45.169138 32672 raft_consensus.cc:2781] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:35:45.169184 32672 raft_consensus.cc:482] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:35:45.169199 32672 raft_consensus.cc:3037] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:45.170020 32672 raft_consensus.cc:504] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:45.170194 32672 leader_election.cc:288] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [CANDIDATE]: Term 1 election: Requested vote from peers 9c4ba2a2341a4302b465a81bca95ec20 (127.29.142.130:36105), 14fd1d18cf7b4488815a7e54dacb663f (127.29.142.131:38391)
I20260806 06:35:45.170390 32361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56ea3b389ace42849ae3f0405b3eb564" candidate_uuid: "8c96a2c142bf4a5bb31f9509e274400d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c4ba2a2341a4302b465a81bca95ec20"
I20260806 06:35:45.170390 32559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56ea3b389ace42849ae3f0405b3eb564" candidate_uuid: "8c96a2c142bf4a5bb31f9509e274400d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14fd1d18cf7b4488815a7e54dacb663f"
I20260806 06:35:45.170471 32361 raft_consensus.cc:3037] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:45.170471 32559 raft_consensus.cc:3037] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:45.171370 32361 raft_consensus.cc:2454] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c96a2c142bf4a5bb31f9509e274400d in term 1.
I20260806 06:35:45.171370 32559 raft_consensus.cc:2454] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c96a2c142bf4a5bb31f9509e274400d in term 1.
I20260806 06:35:45.171559 32068 leader_election.cc:302] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c96a2c142bf4a5bb31f9509e274400d, 9c4ba2a2341a4302b465a81bca95ec20; no voters:
I20260806 06:35:45.171702 32672 raft_consensus.cc:2781] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:45.171761 32672 raft_consensus.cc:686] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [term 1 LEADER]: Becoming Leader. State: Replica: 8c96a2c142bf4a5bb31f9509e274400d, State: Running, Role: LEADER
I20260806 06:35:45.171838 32672 consensus_queue.cc:237] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } }
I20260806 06:35:45.172268 32705 tablet_replica.cc:439] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: TxnStatusTablet state changed. Reason: New leader 8c96a2c142bf4a5bb31f9509e274400d. Latest consensus state: current_term: 1 leader_uuid: "8c96a2c142bf4a5bb31f9509e274400d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } } }
I20260806 06:35:45.172374 32705 tablet_replica.cc:442] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:35:45.172588 31934 catalog_manager.cc:5495] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c96a2c142bf4a5bb31f9509e274400d (127.29.142.129). New cstate: current_term: 1 leader_uuid: "8c96a2c142bf4a5bb31f9509e274400d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } health_report { overall_health: UNKNOWN } } }
I20260806 06:35:45.172652 32722 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:35:45.172771 32722 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:35:45.182560 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.179471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:35711 (local address 127.0.0.1:58364)
0806 06:35:45.179569 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:45.179585 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:45.179649 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:45.179881 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:45.179882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:45.179883 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:35:45.179933 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:45.179934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.180896 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.180897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:45.181468 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:45.181470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.181971 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.181972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:45.182020 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.182039 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:35:45.182409 (+ 370us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:35:45.182416 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:35:45.182453 (+ 37us) client_negotiation.cc:240] Negotiation successful
0806 06:35:45.182498 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 06:35:45.182569 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.179582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58364 (local address 127.29.142.129:35711)
0806 06:35:45.179659 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:45.179661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:45.179697 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:35:45.179749 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:45.179750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:45.179782 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:45.179865 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:45.180003 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.180792 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.181562 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.181881 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.181961 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.182158 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:45.182305 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:45.182413 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:45.182461 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 06:35:45.182483 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:35:45.184437 32559 raft_consensus.cc:1261] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f [term 1 FOLLOWER]: Refusing update from remote peer 8c96a2c142bf4a5bb31f9509e274400d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:45.184544 32361 raft_consensus.cc:1261] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20 [term 1 FOLLOWER]: Refusing update from remote peer 8c96a2c142bf4a5bb31f9509e274400d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:45.184929 32705 consensus_queue.cc:1035] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:45.185084 32672 consensus_queue.cc:1035] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d [LEADER]: Connected to new peer: Peer: permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:45.185778 32668 tablet_replica.cc:439] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f: TxnStatusTablet state changed. Reason: New leader 8c96a2c142bf4a5bb31f9509e274400d. Latest consensus state: current_term: 1 leader_uuid: "8c96a2c142bf4a5bb31f9509e274400d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } } }
I20260806 06:35:45.185895 32668 tablet_replica.cc:442] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:45.185977 32691 tablet_replica.cc:439] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20: TxnStatusTablet state changed. Reason: New leader 8c96a2c142bf4a5bb31f9509e274400d. Latest consensus state: current_term: 1 leader_uuid: "8c96a2c142bf4a5bb31f9509e274400d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } } }
I20260806 06:35:45.186085 32691 tablet_replica.cc:442] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:45.186596 32705 tablet_replica.cc:439] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c96a2c142bf4a5bb31f9509e274400d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } } }
I20260806 06:35:45.186722 32705 tablet_replica.cc:442] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:35:45.187736 32672 tablet_replica.cc:439] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c96a2c142bf4a5bb31f9509e274400d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } } }
I20260806 06:35:45.187839 32672 tablet_replica.cc:442] T 56ea3b389ace42849ae3f0405b3eb564 P 8c96a2c142bf4a5bb31f9509e274400d: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:35:45.188117 32668 tablet_replica.cc:439] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c96a2c142bf4a5bb31f9509e274400d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } } }
I20260806 06:35:45.188213 32668 tablet_replica.cc:442] T 56ea3b389ace42849ae3f0405b3eb564 P 14fd1d18cf7b4488815a7e54dacb663f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:45.188196 32691 tablet_replica.cc:439] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c96a2c142bf4a5bb31f9509e274400d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c96a2c142bf4a5bb31f9509e274400d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35711 } } peers { permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 } } peers { permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 } } }
I20260806 06:35:45.188284 32691 tablet_replica.cc:442] T 56ea3b389ace42849ae3f0405b3eb564 P 9c4ba2a2341a4302b465a81bca95ec20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:45.191883 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.189331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58368 (local address 127.29.142.129:35711)
0806 06:35:45.189383 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:45.189386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:45.189507 (+ 121us) server_negotiation.cc:407] Connection header received
0806 06:35:45.189551 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:45.189551 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:45.189561 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:45.189606 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:45.189710 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.190214 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.191074 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.191323 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.191390 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.191596 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:45.191632 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:45.191715 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:45.191786 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 06:35:45.191807 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:35:45.197886 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.194747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:35805 (local address 127.29.142.129:35711)
0806 06:35:45.194786 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:45.194787 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:45.194952 (+ 165us) server_negotiation.cc:407] Connection header received
0806 06:35:45.194977 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:45.194978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:45.194988 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:45.195053 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:45.195210 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.195946 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.197000 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.197337 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.197436 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.197617 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:45.197651 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:45.197710 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:45.197763 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 06:35:45.197784 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:35:45.198043 32733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.194714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:35711 (local address 127.29.142.129:35805)
0806 06:35:45.194869 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:45.194880 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:45.194933 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:45.195055 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:45.195056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:45.195057 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:35:45.195138 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:45.195139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.196034 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.196035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:45.196904 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:45.196907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.197439 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.197440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:45.197511 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.197528 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:35:45.197721 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:35:45.197723 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:35:45.197758 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 06:35:45.197793 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260806 06:35:45.200574 32361 raft_consensus.cc:1261] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20 [term 1 FOLLOWER]: Refusing update from remote peer 8c96a2c142bf4a5bb31f9509e274400d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:45.200572 32559 raft_consensus.cc:1261] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f [term 1 FOLLOWER]: Refusing update from remote peer 8c96a2c142bf4a5bb31f9509e274400d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:45.200809 32672 consensus_queue.cc:1035] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [LEADER]: Connected to new peer: Peer: permanent_uuid: "14fd1d18cf7b4488815a7e54dacb663f" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 38391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:45.200915 32705 consensus_queue.cc:1035] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c4ba2a2341a4302b465a81bca95ec20" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 36105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:45.220331 31931 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:35:45.220503 31931 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:35:45.221241 31931 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86a4168c19b8472dadee7ff4ce3cc0da: Sending DeleteTablet for 3 replicas of tablet d40685d2c6af48aaa63f66d246bfd2ad
I20260806 06:35:45.221520 32318 tablet_service.cc:1514] Processing DeleteTablet for tablet d40685d2c6af48aaa63f66d246bfd2ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:45 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:47216
I20260806 06:35:45.221544 32131 tablet_service.cc:1514] Processing DeleteTablet for tablet d40685d2c6af48aaa63f66d246bfd2ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:45 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:58356
I20260806 06:35:45.221633 32698 tablet_replica.cc:330] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20: stopping tablet replica
I20260806 06:35:45.221626 32525 tablet_service.cc:1514] Processing DeleteTablet for tablet d40685d2c6af48aaa63f66d246bfd2ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:45 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:43098
I20260806 06:35:45.221706 32698 raft_consensus.cc:2229] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:35:45.221732 32699 tablet_replica.cc:330] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d: stopping tablet replica
I20260806 06:35:45.221755 32698 raft_consensus.cc:2258] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:45.221738 32694 tablet_replica.cc:330] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f: stopping tablet replica
I20260806 06:35:45.221792 32699 raft_consensus.cc:2229] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:35:45.221805 32694 raft_consensus.cc:2229] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:35:45.221848 32694 raft_consensus.cc:2258] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:45.221904 32699 raft_consensus.cc:2258] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:45.222182 32698 ts_tablet_manager.cc:1905] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:45.222224 32694 ts_tablet_manager.cc:1905] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:45.222246 32699 ts_tablet_manager.cc:1905] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:45.224182 32698 ts_tablet_manager.cc:1918] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:35:45.224196 32694 ts_tablet_manager.cc:1918] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:35:45.224196 32699 ts_tablet_manager.cc:1918] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:35:45.224252 32698 log.cc:1196] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/d40685d2c6af48aaa63f66d246bfd2ad
I20260806 06:35:45.224254 32694 log.cc:1196] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/d40685d2c6af48aaa63f66d246bfd2ad
I20260806 06:35:45.224254 32699 log.cc:1196] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998137760399-30266-0/minicluster-data/ts-0/wal/wals/d40685d2c6af48aaa63f66d246bfd2ad
I20260806 06:35:45.224505 32699 ts_tablet_manager.cc:1939] T d40685d2c6af48aaa63f66d246bfd2ad P 8c96a2c142bf4a5bb31f9509e274400d: Deleting consensus metadata
I20260806 06:35:45.224505 32694 ts_tablet_manager.cc:1939] T d40685d2c6af48aaa63f66d246bfd2ad P 14fd1d18cf7b4488815a7e54dacb663f: Deleting consensus metadata
I20260806 06:35:45.224539 32698 ts_tablet_manager.cc:1939] T d40685d2c6af48aaa63f66d246bfd2ad P 9c4ba2a2341a4302b465a81bca95ec20: Deleting consensus metadata
I20260806 06:35:45.224865 31910 catalog_manager.cc:4841] TS 14fd1d18cf7b4488815a7e54dacb663f (127.29.142.131:38391): tablet d40685d2c6af48aaa63f66d246bfd2ad (table test-table [id=d8e8b52267f3472eb6a8aff8e2d8b3a8]) successfully deleted
I20260806 06:35:45.224867 31916 catalog_manager.cc:4841] TS 9c4ba2a2341a4302b465a81bca95ec20 (127.29.142.130:36105): tablet d40685d2c6af48aaa63f66d246bfd2ad (table test-table [id=d8e8b52267f3472eb6a8aff8e2d8b3a8]) successfully deleted
I20260806 06:35:45.225064 31911 catalog_manager.cc:4841] TS 8c96a2c142bf4a5bb31f9509e274400d (127.29.142.129:35711): tablet d40685d2c6af48aaa63f66d246bfd2ad (table test-table [id=d8e8b52267f3472eb6a8aff8e2d8b3a8]) successfully deleted
Aug 06 06:35:45 dist-test-slave-3kk6 krb5kdc[31867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998144, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.142.129@KRBTEST.COM
I20260806 06:35:45.236527 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.223028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58378 (local address 127.29.142.129:35711)
0806 06:35:45.223903 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:45.223906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:45.226891 (+ 2985us) server_negotiation.cc:407] Connection header received
0806 06:35:45.226956 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:45.226957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:45.226970 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:45.227033 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:45.227615 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.228353 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.229181 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.229493 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.229570 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.232840 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:45.232854 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:45.232854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:45.232863 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:45.235100 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:45.235534 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:45.235539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:45.235540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:45.235612 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:45.235818 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:45.235819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:45.235819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:45.236014 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:45.236232 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:45.236275 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 06:35:45.236363 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845}
W20260806 06:35:45.236896 32225 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58378
I20260806 06:35:45.244926 32715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:45.238113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60058 (local address 127.29.142.190:42391)
0806 06:35:45.238169 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:45.238171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:45.238233 (+ 62us) server_negotiation.cc:407] Connection header received
0806 06:35:45.238312 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:45.238313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:45.238325 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:45.238384 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:45.239084 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.239607 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.240366 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:45.240609 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:45.240698 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:45.242094 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:45.242096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:45.242096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:45.242105 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:45.243624 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:45.244006 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:45.244008 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:45.244008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:45.244074 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:45.244352 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:45.244353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:45.244353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:45.244494 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:45.244615 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:45.244716 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 06:35:45.244812 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 06:35:45.245237 31931 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60058
I20260806 06:35:45.246116 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 32042
I20260806 06:35:45.260591 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 32249
I20260806 06:35:45.274159 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 32438
I20260806 06:35:45.288825 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 31889
2026-08-06T06:35:45Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2819 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:45 dist-test-slave-3kk6 krb5kdc[32749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:45 dist-test-slave-3kk6 krb5kdc[32749](info): set up 2 sockets
Aug 06 06:35:45 dist-test-slave-3kk6 krb5kdc[32749](info): commencing operation
krb5kdc: starting...
W20260806 06:35:46.535076 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:35:46 dist-test-slave-3kk6 krb5kdc[32749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:35:46.666591 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:41341
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:44701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:41341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:46.752144 32767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:46.752455 32767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:46.752501 32767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:46.753762 32767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:46.753865 32767 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:35:46.753890 32767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:46.753912 32767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:46.753942 32767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:46.753962 32767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:46.755661 32767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:44701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:41341
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:41341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:46.756011 32767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:46.756291 32767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:46.761806 305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:46.761818 306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:46.761898 309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:46.762190 32767 server_base.cc:1034] running on GCE node
I20260806 06:35:46.762532 32767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:46.762914 32767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:46.764109 32767 hybrid_clock.cc:648] HybridClock initialized: now 1785998146764094 us; error 53 us; skew 500 ppm
Aug 06 06:35:46 dist-test-slave-3kk6 krb5kdc[32749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:46.771836 32767 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:35:46.772428 32767 webserver.cc:460] Webserver started at http://127.29.142.190:35861/ using document root <none> and password file <none>
I20260806 06:35:46.772641 32767 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:46.772684 32767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:46.772796 32767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:46.773957 32767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "6f8a86b6af4b4dd5a790c825e478b47e"
format_stamp: "Formatted at 2026-08-06 06:35:46 on dist-test-slave-3kk6"
server_key: "e158658b426b96f4a995ac659bf57793"
server_key_iv: "79f7457a879686e85cc76a7f31646d9b"
server_key_version: "encryptionkey@0"
I20260806 06:35:46.774315 32767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "6f8a86b6af4b4dd5a790c825e478b47e"
format_stamp: "Formatted at 2026-08-06 06:35:46 on dist-test-slave-3kk6"
server_key: "e158658b426b96f4a995ac659bf57793"
server_key_iv: "79f7457a879686e85cc76a7f31646d9b"
server_key_version: "encryptionkey@0"
I20260806 06:35:46.775940 32767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:46.776748 317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:46.777000 32767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:35:46.777127 32767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "6f8a86b6af4b4dd5a790c825e478b47e"
format_stamp: "Formatted at 2026-08-06 06:35:46 on dist-test-slave-3kk6"
server_key: "e158658b426b96f4a995ac659bf57793"
server_key_iv: "79f7457a879686e85cc76a7f31646d9b"
server_key_version: "encryptionkey@0"
I20260806 06:35:46.777220 32767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:46.787693 32767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:46.789804 32767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:46.790010 32767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:46.795352 32767 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:41341
I20260806 06:35:46.795466 415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:41341 every 8 connection(s)
I20260806 06:35:46.795843 32767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:46.796594 416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:46.799202 416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e: Bootstrap starting.
I20260806 06:35:46.799853 416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:46.800154 416 log.cc:824] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:46.800959 416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e: No bootstrap required, opened a new log
I20260806 06:35:46.802935 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 32767
I20260806 06:35:46.803050 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:35:46.803234 30266 external_mini_cluster.cc:1428] Setting key cb724fa16841bcde83bf864fb1df5db9
I20260806 06:35:46.803368 416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8a86b6af4b4dd5a790c825e478b47e" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41341 } }
I20260806 06:35:46.803604 416 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:46.803653 416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8a86b6af4b4dd5a790c825e478b47e, State: Initialized, Role: FOLLOWER
I20260806 06:35:46.803916 416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8a86b6af4b4dd5a790c825e478b47e" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41341 } }
I20260806 06:35:46.804040 416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:46.804104 416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:46.804163 416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:46.804901 416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8a86b6af4b4dd5a790c825e478b47e" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41341 } }
I20260806 06:35:46.805034 416 leader_election.cc:302] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f8a86b6af4b4dd5a790c825e478b47e; no voters:
I20260806 06:35:46.805238 416 leader_election.cc:288] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:46.805300 422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:46.805506 422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [term 1 LEADER]: Becoming Leader. State: Replica: 6f8a86b6af4b4dd5a790c825e478b47e, State: Running, Role: LEADER
I20260806 06:35:46.805522 416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:46.805610 422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8a86b6af4b4dd5a790c825e478b47e" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41341 } }
I20260806 06:35:46.805925 422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f8a86b6af4b4dd5a790c825e478b47e. Latest consensus state: current_term: 1 leader_uuid: "6f8a86b6af4b4dd5a790c825e478b47e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8a86b6af4b4dd5a790c825e478b47e" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41341 } } }
I20260806 06:35:46.806022 422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:46.805939 423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f8a86b6af4b4dd5a790c825e478b47e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8a86b6af4b4dd5a790c825e478b47e" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41341 } } }
I20260806 06:35:46.806188 423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:46.806347 431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:46.807207 431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:46.809617 431 catalog_manager.cc:1347] Generated new cluster ID: 3ba30e0cbd434d13a4ac1c47b1f65cda
I20260806 06:35:46.809680 431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:35:46 dist-test-slave-3kk6 krb5kdc[32749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:46.816058 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:46.803937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57056 (local address 127.29.142.190:41341)
0806 06:35:46.804194 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:46.804205 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:46.804241 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:35:46.804368 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:46.804374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:46.804420 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:46.804533 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:46.805416 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:46.807053 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:46.808068 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:46.808659 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:46.808785 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:46.811943 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:46.811947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:46.811947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:46.811959 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:46.814192 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:46.814678 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:46.814688 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:46.814688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:46.814808 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:46.815053 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:46.815054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:46.815054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:46.815363 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:46.815516 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:46.815688 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 06:35:46.815834 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":115,"threads_started":1}
I20260806 06:35:46.821237 431 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:46.821936 431 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:46.824671 431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e: Generated new TSK 0
I20260806 06:35:46.824888 431 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:35:46.861637 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41341
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:44701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:46.952908 459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:46.953141 459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:46.953169 459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:46.954368 459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:46.954416 459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:46.954438 459 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:35:46.954450 459 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:35:46.954468 459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:35:46.955991 459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:44701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.142.190:41341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:46.956255 459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:46.956525 459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:46.962026 469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:46.962023 466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:46.962023 467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:46.962395 459 server_base.cc:1034] running on GCE node
I20260806 06:35:46.962618 459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:46.962940 459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:46.964109 459 hybrid_clock.cc:648] HybridClock initialized: now 1785998146964085 us; error 42 us; skew 500 ppm
Aug 06 06:35:46 dist-test-slave-3kk6 krb5kdc[32749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:46.966986 459 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:35:46.967542 459 webserver.cc:460] Webserver started at http://127.29.142.129:46711/ using document root <none> and password file <none>
I20260806 06:35:46.967798 459 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:46.967862 459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:46.967998 459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:46.969147 459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "8f167c29d7bc4c1faaf2696c72e35658"
format_stamp: "Formatted at 2026-08-06 06:35:46 on dist-test-slave-3kk6"
server_key: "837d6f54b404f55353159bf0c5e13cf7"
server_key_iv: "9368652648b3a45021c711169169c99b"
server_key_version: "encryptionkey@0"
I20260806 06:35:46.969542 459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "8f167c29d7bc4c1faaf2696c72e35658"
format_stamp: "Formatted at 2026-08-06 06:35:46 on dist-test-slave-3kk6"
server_key: "837d6f54b404f55353159bf0c5e13cf7"
server_key_iv: "9368652648b3a45021c711169169c99b"
server_key_version: "encryptionkey@0"
I20260806 06:35:46.971040 459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:46.971876 479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:46.972074 459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:46.972188 459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "8f167c29d7bc4c1faaf2696c72e35658"
format_stamp: "Formatted at 2026-08-06 06:35:46 on dist-test-slave-3kk6"
server_key: "837d6f54b404f55353159bf0c5e13cf7"
server_key_iv: "9368652648b3a45021c711169169c99b"
server_key_version: "encryptionkey@0"
I20260806 06:35:46.972276 459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:46.989462 459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:46.991855 459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:46.992046 459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:46.992621 459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:46.992704 459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:46.992774 459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:46.992806 459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:35:46 dist-test-slave-3kk6 krb5kdc[32749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:47.000890 459 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:44773
I20260806 06:35:47.001004 639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:44773 every 8 connection(s)
I20260806 06:35:47.001374 459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:35:47.003357 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:46.993341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:56295 (local address 127.29.142.190:41341)
0806 06:35:46.993423 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:46.993426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:46.993682 (+ 256us) server_negotiation.cc:407] Connection header received
0806 06:35:46.994489 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:46.994491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:46.994509 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:46.994589 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:46.995672 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:46.996276 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:46.997390 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:46.997622 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:46.997714 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.000221 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:47.000223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:47.000224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:47.000233 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:47.002051 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.002444 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.002444 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.002445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.002504 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.002789 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.002790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.002791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.002913 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:47.002981 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.003224 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.003291 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 06:35:47.003587 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:46.993305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41341 (local address 127.29.142.129:56295)
0806 06:35:46.993521 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:46.993574 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:46.994358 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:46.994739 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:46.994744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:46.995207 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:46.995532 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:46.995540 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:46.996431 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:46.996433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:46.997273 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:46.997275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:46.997738 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:46.997739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:46.997872 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:46.998476 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:46.998485 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:47.000102 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:47.002127 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.002132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.002133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.002369 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.002567 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.002567 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.002568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.002729 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.002976 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:47.002981 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:47.003102 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.003182 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.003311 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":111,"threads_started":1}
I20260806 06:35:47.004246 640 heartbeater.cc:347] Connected to a master server at 127.29.142.190:41341
I20260806 06:35:47.004326 640 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:47.004530 640 heartbeater.cc:511] Master 127.29.142.190:41341 requested a full tablet report, sending...
I20260806 06:35:47.005263 349 ts_manager.cc:194] Registered new tserver with Master: 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129:44773)
I20260806 06:35:47.006191 349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:56295
I20260806 06:35:47.008534 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 459
I20260806 06:35:47.008662 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:35:47.008841 30266 external_mini_cluster.cc:1428] Setting key a957457e9e2edf79793fb1daefcb16dd
I20260806 06:35:47.012132 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.005563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:36445 (local address 127.29.142.190:41341)
0806 06:35:47.005630 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.005632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.005764 (+ 132us) server_negotiation.cc:407] Connection header received
0806 06:35:47.005792 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.005793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.005803 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.005874 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:47.006220 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.006894 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.007721 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.007907 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.007964 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.009109 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:47.009111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:47.009111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:47.009118 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:47.010909 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.011266 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.011267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.011268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.011331 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.011542 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.011543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.011544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.011668 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:47.011784 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.011958 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.012044 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:35:47.012141 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.005473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41341 (local address 127.29.142.129:36445)
0806 06:35:47.005656 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.005702 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.005756 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.005888 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.005888 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.006042 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:47.006097 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.006099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.006997 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.006998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.007644 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.007646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.007978 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.007978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.008037 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.008561 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:47.008567 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:47.009003 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:47.010994 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.010995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.010995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.011169 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.011410 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.011411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.011411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.011476 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.011786 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:47.011787 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:47.011837 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.011919 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.012036 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:35:47.038714 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41341
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:44701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:47.125568 653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:47.125808 653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:47.125835 653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:47.127117 653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:47.127173 653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:47.127195 653 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:35:47.127208 653 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:35:47.127270 653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:35:47.128834 653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:44701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.142.190:41341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:47.129132 653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:47.129400 653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:47.135705 666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:47.135720 661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:47.135705 663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:47.136174 653 server_base.cc:1034] running on GCE node
I20260806 06:35:47.136410 653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:47.136740 653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:47.137929 653 hybrid_clock.cc:648] HybridClock initialized: now 1785998147137933 us; error 54 us; skew 500 ppm
Aug 06 06:35:47 dist-test-slave-3kk6 krb5kdc[32749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998147, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:47.140950 653 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:35:47.141561 653 webserver.cc:460] Webserver started at http://127.29.142.130:38615/ using document root <none> and password file <none>
I20260806 06:35:47.141786 653 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:47.141834 653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:47.141943 653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:47.143164 653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "8ea4545546aa4df2b96ddaeed3243847"
format_stamp: "Formatted at 2026-08-06 06:35:47 on dist-test-slave-3kk6"
server_key: "3b10076746a57445e020fd6631a29d20"
server_key_iv: "a6325e8a77735f06598554fe696831ff"
server_key_version: "encryptionkey@0"
I20260806 06:35:47.143498 653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "8ea4545546aa4df2b96ddaeed3243847"
format_stamp: "Formatted at 2026-08-06 06:35:47 on dist-test-slave-3kk6"
server_key: "3b10076746a57445e020fd6631a29d20"
server_key_iv: "a6325e8a77735f06598554fe696831ff"
server_key_version: "encryptionkey@0"
I20260806 06:35:47.145053 653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:35:47.145877 676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:47.146082 653 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:47.146204 653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "8ea4545546aa4df2b96ddaeed3243847"
format_stamp: "Formatted at 2026-08-06 06:35:47 on dist-test-slave-3kk6"
server_key: "3b10076746a57445e020fd6631a29d20"
server_key_iv: "a6325e8a77735f06598554fe696831ff"
server_key_version: "encryptionkey@0"
I20260806 06:35:47.146294 653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:47.163000 653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:47.165319 653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:47.165519 653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:47.166182 653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:47.166251 653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:47.166301 653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:47.166329 653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:35:47 dist-test-slave-3kk6 krb5kdc[32749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998147, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:47.174285 653 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:41743
I20260806 06:35:47.174371 839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:41743 every 8 connection(s)
I20260806 06:35:47.174803 653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:35:47.177639 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.166701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:60441 (local address 127.29.142.190:41341)
0806 06:35:47.166765 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.166768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.166995 (+ 227us) server_negotiation.cc:407] Connection header received
0806 06:35:47.167512 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.167513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.167529 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.167623 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:47.168429 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.168948 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.170792 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.171025 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.171125 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.174294 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:47.174296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:47.174296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:47.174307 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:47.176092 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.176528 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.176530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.176530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.176626 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.176925 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.176926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.176927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.177063 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:47.177144 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.177449 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.177540 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:35:47.177856 686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.166648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41341 (local address 127.29.142.130:60441)
0806 06:35:47.166837 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.166881 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.167401 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.167678 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.167682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.168097 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:47.168309 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.168315 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.169070 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.169071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.170618 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.170623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.171146 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.171147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.171298 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.172002 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:47.172011 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:47.174132 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:47.176191 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.176198 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.176201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.176437 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.176715 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.176716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.176716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.176848 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.177146 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:47.177153 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:47.177305 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.177424 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.177598 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260806 06:35:47.178445 840 heartbeater.cc:347] Connected to a master server at 127.29.142.190:41341
I20260806 06:35:47.178541 840 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:47.178833 840 heartbeater.cc:511] Master 127.29.142.190:41341 requested a full tablet report, sending...
I20260806 06:35:47.179272 348 ts_manager.cc:194] Registered new tserver with Master: 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130:41743)
I20260806 06:35:47.179737 348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:60441
I20260806 06:35:47.185074 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 653
I20260806 06:35:47.185221 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:35:47.185453 30266 external_mini_cluster.cc:1428] Setting key 113a2d4d6c8f5e6fca0ad74c1b88b70a
I20260806 06:35:47.185842 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.179875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:55613 (local address 127.29.142.190:41341)
0806 06:35:47.179920 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.179922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.180072 (+ 150us) server_negotiation.cc:407] Connection header received
0806 06:35:47.180153 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.180154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.180165 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.180229 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:47.180552 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.181110 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.181856 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.182032 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.182082 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.183163 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:47.183164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:47.183165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:47.183171 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:47.184504 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.184868 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.184870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.184870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.184945 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.185150 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.185151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.185151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.185397 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:47.185475 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.185632 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.185720 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:35:47.185863 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.179785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41341 (local address 127.29.142.130:55613)
0806 06:35:47.179995 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.180006 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.180068 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.180235 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.180235 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.180412 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:47.180472 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.180474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.181227 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.181228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.181777 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.181779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.182101 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.182102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.182153 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.182736 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:47.182742 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:47.183058 (+ 316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:47.184589 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.184590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.184590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.184765 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.185021 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.185022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.185022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.185080 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.185461 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:47.185462 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:47.185530 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.185599 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.185742 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:35:47.214202 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41341
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:44701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:47.301579 853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:47.301818 853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:47.301847 853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:47.303009 853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:47.303049 853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:47.303071 853 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:35:47.303092 853 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:35:47.303109 853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:35:47.305228 853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:44701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.142.190:41341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:47.305505 853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:47.305763 853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:47.311595 865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:47.311595 862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:47.311618 861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:47.311928 853 server_base.cc:1034] running on GCE node
I20260806 06:35:47.312181 853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:47.312453 853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:47.313637 853 hybrid_clock.cc:648] HybridClock initialized: now 1785998147313622 us; error 38 us; skew 500 ppm
Aug 06 06:35:47 dist-test-slave-3kk6 krb5kdc[32749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998147, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:47.316560 853 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:35:47.317173 853 webserver.cc:460] Webserver started at http://127.29.142.131:42481/ using document root <none> and password file <none>
I20260806 06:35:47.317436 853 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:47.317488 853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:47.317632 853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:47.318879 853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "b1854ee34f734988be91601bb216ca40"
format_stamp: "Formatted at 2026-08-06 06:35:47 on dist-test-slave-3kk6"
server_key: "25279ee85e2015210c35d73da757ad46"
server_key_iv: "93de868c72027ced0c74e3133d3211bc"
server_key_version: "encryptionkey@0"
I20260806 06:35:47.319281 853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "b1854ee34f734988be91601bb216ca40"
format_stamp: "Formatted at 2026-08-06 06:35:47 on dist-test-slave-3kk6"
server_key: "25279ee85e2015210c35d73da757ad46"
server_key_iv: "93de868c72027ced0c74e3133d3211bc"
server_key_version: "encryptionkey@0"
I20260806 06:35:47.320858 853 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:35:47.321758 876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:47.322045 853 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:35:47.322156 853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "b1854ee34f734988be91601bb216ca40"
format_stamp: "Formatted at 2026-08-06 06:35:47 on dist-test-slave-3kk6"
server_key: "25279ee85e2015210c35d73da757ad46"
server_key_iv: "93de868c72027ced0c74e3133d3211bc"
server_key_version: "encryptionkey@0"
I20260806 06:35:47.322242 853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:47.339588 853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:47.342231 853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:47.342449 853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:47.343223 853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:47.343304 853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:47.343350 853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:47.343372 853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:35:47 dist-test-slave-3kk6 krb5kdc[32749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998147, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:47.351625 853 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:43683
I20260806 06:35:47.351747 1034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:43683 every 8 connection(s)
I20260806 06:35:47.352111 853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:35:47.354321 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.344028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:56625 (local address 127.29.142.190:41341)
0806 06:35:47.344122 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.344125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.344141 (+ 16us) server_negotiation.cc:407] Connection header received
0806 06:35:47.344769 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.344771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.344785 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.344853 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:47.345580 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.346142 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.347434 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.347636 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.347740 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.350919 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:47.350921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:47.350921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:47.350930 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:47.352771 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.353250 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.353252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.353252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.353328 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.353601 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.353602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.353602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.353734 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:47.353820 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.354125 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.354219 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 06:35:47.354502 885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.343663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41341 (local address 127.29.142.131:56625)
0806 06:35:47.343864 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.343909 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.344642 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.344934 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.344939 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.345252 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:47.345462 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.345468 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.346255 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.346256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.347329 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.347332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.347751 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.347752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.347893 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.348586 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:47.348600 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:47.350782 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:47.352869 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.352876 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.352878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.353162 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.353410 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.353410 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.353411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.353528 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.353818 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:47.353826 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:47.353973 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.354088 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.354259 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":129,"threads_started":1}
I20260806 06:35:47.355072 1037 heartbeater.cc:347] Connected to a master server at 127.29.142.190:41341
I20260806 06:35:47.355166 1037 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:47.355365 1037 heartbeater.cc:511] Master 127.29.142.190:41341 requested a full tablet report, sending...
I20260806 06:35:47.355777 348 ts_manager.cc:194] Registered new tserver with Master: b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683)
I20260806 06:35:47.356362 348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:56625
I20260806 06:35:47.360884 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 853
I20260806 06:35:47.360982 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:35:47.361202 30266 external_mini_cluster.cc:1428] Setting key 0f0db4c2740a3f0b261ffd178d7d876c
I20260806 06:35:47.362309 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:35:47.362612 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.356402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:47553 (local address 127.29.142.190:41341)
0806 06:35:47.356488 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.356490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.356500 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:35:47.356563 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.356563 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.356574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.356651 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:47.356983 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.357540 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.358265 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.358563 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.358635 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.359616 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:47.359618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:47.359618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:47.359624 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:47.361214 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.361700 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.361701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.361701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.361766 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.362030 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.362030 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.362031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.362171 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:47.362246 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.362461 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.362551 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 06:35:47.362613 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.356275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41341 (local address 127.29.142.131:47553)
0806 06:35:47.356428 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.356441 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.356494 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.356669 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.356669 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.356842 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:47.356898 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.356900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.357634 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.357634 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.358155 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.358156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.358709 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.358710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.358805 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.359226 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:47.359231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:47.359523 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:47.361328 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.361329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.361330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.361597 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.361871 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.361872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.361873 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.361954 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.362280 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:47.362281 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:47.362337 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.362414 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.362521 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
Aug 06 06:35:47 dist-test-slave-3kk6 krb5kdc[32749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:35:47 dist-test-slave-3kk6 krb5kdc[32749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:47.388089 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.380318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57060 (local address 127.29.142.190:41341)
0806 06:35:47.380381 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.380383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.380393 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:35:47.380431 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.380432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.380441 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.380498 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:47.381212 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.381801 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.382565 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.382889 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.382955 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.385458 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:47.385460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:47.385460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:47.385468 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:47.386853 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.387224 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.387225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.387225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.387288 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.387502 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.387503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.387504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.387625 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:47.387734 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.387886 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.387975 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:35:47.389977 348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:35:47.391762 348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:35:47.398676 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.394578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:44773 (local address 127.0.0.1:58400)
0806 06:35:47.394863 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.394873 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.394922 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.395493 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.395494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.395496 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:47.395530 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.395531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.396693 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.396694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.397611 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.397614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.398333 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.398334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.398414 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.398429 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.398502 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.398562 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":119,"threads_started":1}
I20260806 06:35:47.398819 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.394873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58002 (local address 127.29.142.131:43683)
0806 06:35:47.395060 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.395064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.395074 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:35:47.395109 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.395112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.395125 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.395203 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:47.395360 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.396178 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.397643 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.398388 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.398471 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.398533 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.398615 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.398699 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":106,"threads_started":1}
I20260806 06:35:47.398939 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.394881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58400 (local address 127.29.142.129:44773)
0806 06:35:47.395342 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.395346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.395357 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:35:47.395396 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.395399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.395413 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.395495 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:47.395591 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.396630 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.397697 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.398232 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.398337 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.398401 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.398769 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.398816 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":101,"threads_started":1}
I20260806 06:35:47.398978 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.394336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:41743 (local address 127.0.0.1:55240)
0806 06:35:47.394475 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.394522 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.394587 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.395030 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.395033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.395036 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:47.395127 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.395129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.396627 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.396628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.397673 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.397675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.398545 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.398546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.398614 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.398619 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.398677 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.398718 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":87,"threads_started":1}
I20260806 06:35:47.399243 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.394340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:43683 (local address 127.0.0.1:58002)
0806 06:35:47.394502 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.394537 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.394595 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.395218 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.395219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.395222 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:47.395286 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.395289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.396284 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.396285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.397546 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.397549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.398463 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.398464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.398538 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.398540 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.398598 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.398630 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260806 06:35:47.399418 1070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.394552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55240 (local address 127.29.142.130:41743)
0806 06:35:47.394829 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.394836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.394848 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:35:47.394897 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.394899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.394916 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.395209 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:47.395400 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.396532 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.397749 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.398447 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.399177 (+ 730us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.399247 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.399274 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.399317 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":183,"threads_started":1}
I20260806 06:35:47.399708 551 tablet_service.cc:1467] Processing CreateTablet for tablet 59e4741c155344f789fc16ffb10e5a5e (DEFAULT_TABLE table=test-table [id=46d12391c369428dbe19e9e83ac224be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:47.400120 551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59e4741c155344f789fc16ffb10e5a5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:47.399989 942 tablet_service.cc:1467] Processing CreateTablet for tablet 59e4741c155344f789fc16ffb10e5a5e (DEFAULT_TABLE table=test-table [id=46d12391c369428dbe19e9e83ac224be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:47.400087 753 tablet_service.cc:1467] Processing CreateTablet for tablet 59e4741c155344f789fc16ffb10e5a5e (DEFAULT_TABLE table=test-table [id=46d12391c369428dbe19e9e83ac224be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:47.400403 942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59e4741c155344f789fc16ffb10e5a5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:47.400403 753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59e4741c155344f789fc16ffb10e5a5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:47.403683 1075 tablet_bootstrap.cc:492] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658: Bootstrap starting.
I20260806 06:35:47.404358 1075 tablet_bootstrap.cc:654] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:47.404510 1077 tablet_bootstrap.cc:492] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40: Bootstrap starting.
I20260806 06:35:47.404759 1075 log.cc:824] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:47.404937 1079 tablet_bootstrap.cc:492] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847: Bootstrap starting.
I20260806 06:35:47.405520 1077 tablet_bootstrap.cc:654] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:47.405619 1079 tablet_bootstrap.cc:654] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:47.405854 1077 log.cc:824] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:47.405968 1079 log.cc:824] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:47.406634 1077 tablet_bootstrap.cc:492] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40: No bootstrap required, opened a new log
I20260806 06:35:47.406767 1077 ts_tablet_manager.cc:1397] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:35:47.406908 1079 tablet_bootstrap.cc:492] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847: No bootstrap required, opened a new log
I20260806 06:35:47.406982 1079 ts_tablet_manager.cc:1397] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:35:47.407375 1075 tablet_bootstrap.cc:492] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658: No bootstrap required, opened a new log
I20260806 06:35:47.407495 1075 ts_tablet_manager.cc:1397] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 06:35:47.408460 1079 raft_consensus.cc:348] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } }
I20260806 06:35:47.408607 1079 raft_consensus.cc:374] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:47.408629 1079 raft_consensus.cc:729] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea4545546aa4df2b96ddaeed3243847, State: Initialized, Role: FOLLOWER
I20260806 06:35:47.408761 1079 consensus_queue.cc:260] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } }
I20260806 06:35:47.408967 1077 raft_consensus.cc:348] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } }
I20260806 06:35:47.409075 840 heartbeater.cc:503] Master 127.29.142.190:41341 was elected leader, sending a full tablet report...
I20260806 06:35:47.409128 1077 raft_consensus.cc:374] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:47.409179 1077 raft_consensus.cc:729] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1854ee34f734988be91601bb216ca40, State: Initialized, Role: FOLLOWER
I20260806 06:35:47.409333 1077 consensus_queue.cc:260] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } }
I20260806 06:35:47.409615 1077 ts_tablet_manager.cc:1428] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:35:47.409816 1037 heartbeater.cc:503] Master 127.29.142.190:41341 was elected leader, sending a full tablet report...
I20260806 06:35:47.409058 1079 ts_tablet_manager.cc:1428] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:35:47.410332 1075 raft_consensus.cc:348] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } }
I20260806 06:35:47.410483 1075 raft_consensus.cc:374] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:47.410537 1075 raft_consensus.cc:729] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f167c29d7bc4c1faaf2696c72e35658, State: Initialized, Role: FOLLOWER
I20260806 06:35:47.410702 1075 consensus_queue.cc:260] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } }
I20260806 06:35:47.410996 1075 ts_tablet_manager.cc:1428] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:35:47.411034 640 heartbeater.cc:503] Master 127.29.142.190:41341 was elected leader, sending a full tablet report...
W20260806 06:35:47.425526 841 tablet.cc:2369] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:35:47.502204 642 tablet.cc:2369] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:35:47.596688 1085 raft_consensus.cc:482] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:35:47.596866 1085 raft_consensus.cc:504] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } }
I20260806 06:35:47.597282 1085 leader_election.cc:288] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130:41743), b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683)
I20260806 06:35:47.600562 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.597638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:54655 (local address 127.29.142.131:43683)
0806 06:35:47.597698 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.597700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.597856 (+ 156us) server_negotiation.cc:407] Connection header received
0806 06:35:47.597929 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.597930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.597940 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.598027 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:47.598234 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.598740 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.599787 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.600170 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.600232 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.600268 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.600434 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.600483 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:35:47.600585 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.597488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:41743 (local address 127.29.142.129:46749)
0806 06:35:47.597720 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.597734 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.597848 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.598012 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.598014 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.598016 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:47.598103 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.598105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.598895 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.598896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.599674 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.599676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.600307 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.600309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.600368 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.600372 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.600453 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.600492 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":131,"threads_started":1}
I20260806 06:35:47.600589 1070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.597595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:46749 (local address 127.29.142.130:41743)
0806 06:35:47.597678 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.597681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.597851 (+ 170us) server_negotiation.cc:407] Connection header received
0806 06:35:47.597914 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.597915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.597927 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.597982 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:47.598197 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.598770 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.599792 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.600179 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.600279 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.600289 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.600481 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.600526 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 06:35:47.600593 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.597490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:43683 (local address 127.29.142.129:54655)
0806 06:35:47.597750 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.597760 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.597862 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.598049 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.598050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.598053 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:47.598122 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.598124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.598827 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.598829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.599678 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.599681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.600268 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.600270 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.600354 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.600358 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.600403 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.600476 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":34,"thread_start_us":186,"threads_started":1}
I20260806 06:35:47.601018 778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59e4741c155344f789fc16ffb10e5a5e" candidate_uuid: "8f167c29d7bc4c1faaf2696c72e35658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea4545546aa4df2b96ddaeed3243847" is_pre_election: true
I20260806 06:35:47.601046 970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59e4741c155344f789fc16ffb10e5a5e" candidate_uuid: "8f167c29d7bc4c1faaf2696c72e35658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1854ee34f734988be91601bb216ca40" is_pre_election: true
I20260806 06:35:47.601212 778 raft_consensus.cc:2454] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f167c29d7bc4c1faaf2696c72e35658 in term 0.
I20260806 06:35:47.601218 970 raft_consensus.cc:2454] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f167c29d7bc4c1faaf2696c72e35658 in term 0.
I20260806 06:35:47.601454 480 leader_election.cc:302] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea4545546aa4df2b96ddaeed3243847, 8f167c29d7bc4c1faaf2696c72e35658; no voters:
I20260806 06:35:47.601596 1085 raft_consensus.cc:2781] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:35:47.601660 1085 raft_consensus.cc:482] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:35:47.601684 1085 raft_consensus.cc:3037] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:47.602468 1085 raft_consensus.cc:504] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } }
I20260806 06:35:47.602643 1085 leader_election.cc:288] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [CANDIDATE]: Term 1 election: Requested vote from peers 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130:41743), b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683)
W20260806 06:35:47.602785 1038 tablet.cc:2369] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:35:47.602840 970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59e4741c155344f789fc16ffb10e5a5e" candidate_uuid: "8f167c29d7bc4c1faaf2696c72e35658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1854ee34f734988be91601bb216ca40"
I20260806 06:35:47.602954 970 raft_consensus.cc:3037] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:47.602981 778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59e4741c155344f789fc16ffb10e5a5e" candidate_uuid: "8f167c29d7bc4c1faaf2696c72e35658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea4545546aa4df2b96ddaeed3243847"
I20260806 06:35:47.603052 778 raft_consensus.cc:3037] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:47.603945 970 raft_consensus.cc:2454] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f167c29d7bc4c1faaf2696c72e35658 in term 1.
I20260806 06:35:47.603945 778 raft_consensus.cc:2454] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f167c29d7bc4c1faaf2696c72e35658 in term 1.
I20260806 06:35:47.604200 480 leader_election.cc:302] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea4545546aa4df2b96ddaeed3243847, 8f167c29d7bc4c1faaf2696c72e35658; no voters:
I20260806 06:35:47.604312 1085 raft_consensus.cc:2781] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:47.604460 1085 raft_consensus.cc:686] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 LEADER]: Becoming Leader. State: Replica: 8f167c29d7bc4c1faaf2696c72e35658, State: Running, Role: LEADER
I20260806 06:35:47.604568 1085 consensus_queue.cc:237] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } }
I20260806 06:35:47.605507 349 catalog_manager.cc:5495] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129). New cstate: current_term: 1 leader_uuid: "8f167c29d7bc4c1faaf2696c72e35658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } health_report { overall_health: HEALTHY } } }
Aug 06 06:35:47 dist-test-slave-3kk6 krb5kdc[32749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998146, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:47.655457 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.647567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57062 (local address 127.29.142.190:41341)
0806 06:35:47.647655 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.647658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.647812 (+ 154us) server_negotiation.cc:407] Connection header received
0806 06:35:47.647910 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.647911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.647923 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.648000 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:47.648545 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.649111 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.649911 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.650095 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.650180 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.652698 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:47.652699 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:47.652700 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:47.652708 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:47.654036 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.654470 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.654472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.654472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.654529 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:47.654788 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:47.654789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:47.654790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:47.654907 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:47.654993 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.655164 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.655258 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 06:35:47.655516 1096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.647525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41341 (local address 127.0.0.1:57062)
0806 06:35:47.647717 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.647727 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.647830 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.648019 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.648020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.648384 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:47.648441 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.648444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.649205 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.649206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.649805 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.649808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.650200 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.650201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.650262 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.651005 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:47.651010 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:47.652580 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:47.654148 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.654158 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.654161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.654372 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.654602 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:47.654603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:47.654604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:47.654709 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:47.654983 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:47.654992 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:47.655040 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.655140 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.655335 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":89,"thread_start_us":107,"threads_started":1}
I20260806 06:35:47.658550 349 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:57062:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:35:47.658819 349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:35:47.660769 753 tablet_service.cc:1467] Processing CreateTablet for tablet 2277dec49cb7408b81b4fb22934d314c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b37b90cae824e4ab59e8e5e77c49510]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:35:47.660773 551 tablet_service.cc:1467] Processing CreateTablet for tablet 2277dec49cb7408b81b4fb22934d314c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b37b90cae824e4ab59e8e5e77c49510]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:35:47.660982 551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2277dec49cb7408b81b4fb22934d314c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:47.660981 753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2277dec49cb7408b81b4fb22934d314c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:47.662798 1075 tablet_bootstrap.cc:492] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658: Bootstrap starting.
I20260806 06:35:47.662838 942 tablet_service.cc:1467] Processing CreateTablet for tablet 2277dec49cb7408b81b4fb22934d314c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1b37b90cae824e4ab59e8e5e77c49510]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:35:47.662916 1079 tablet_bootstrap.cc:492] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: Bootstrap starting.
I20260806 06:35:47.663017 942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2277dec49cb7408b81b4fb22934d314c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:47.663264 1075 tablet_bootstrap.cc:654] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:47.663654 1079 tablet_bootstrap.cc:654] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:47.663995 1075 tablet_bootstrap.cc:492] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658: No bootstrap required, opened a new log
I20260806 06:35:47.664048 1075 ts_tablet_manager.cc:1397] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:47.664463 1075 raft_consensus.cc:348] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.664572 1075 raft_consensus.cc:374] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:47.664595 1075 raft_consensus.cc:729] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f167c29d7bc4c1faaf2696c72e35658, State: Initialized, Role: FOLLOWER
I20260806 06:35:47.664602 1079 tablet_bootstrap.cc:492] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: No bootstrap required, opened a new log
I20260806 06:35:47.664646 1079 ts_tablet_manager.cc:1397] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:35:47.664645 1075 consensus_queue.cc:260] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.664749 1075 ts_tablet_manager.cc:1428] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:47.664825 1079 raft_consensus.cc:348] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.664889 1079 raft_consensus.cc:374] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:47.664911 1079 raft_consensus.cc:729] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea4545546aa4df2b96ddaeed3243847, State: Initialized, Role: FOLLOWER
I20260806 06:35:47.664875 1085 tablet_replica.cc:439] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } }
I20260806 06:35:47.664970 1079 consensus_queue.cc:260] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.664997 1085 tablet_replica.cc:442] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:47.665078 1079 ts_tablet_manager.cc:1428] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:47.665163 1077 tablet_bootstrap.cc:492] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40: Bootstrap starting.
I20260806 06:35:47.665187 1083 tablet_replica.cc:439] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } }
I20260806 06:35:47.665293 1083 tablet_replica.cc:442] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:47.665835 1077 tablet_bootstrap.cc:654] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:47.666628 1077 tablet_bootstrap.cc:492] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40: No bootstrap required, opened a new log
I20260806 06:35:47.666800 1077 ts_tablet_manager.cc:1397] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:35:47.666965 1077 raft_consensus.cc:348] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.667033 1077 raft_consensus.cc:374] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:47.667047 1077 raft_consensus.cc:729] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1854ee34f734988be91601bb216ca40, State: Initialized, Role: FOLLOWER
I20260806 06:35:47.667112 1077 consensus_queue.cc:260] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.667204 1077 ts_tablet_manager.cc:1428] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:47.667346 1084 tablet_replica.cc:439] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } }
I20260806 06:35:47.667450 1084 tablet_replica.cc:442] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:47.781221 1083 raft_consensus.cc:482] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:35:47.781349 1083 raft_consensus.cc:504] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.781755 1083 leader_election.cc:288] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129:44773), b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683)
I20260806 06:35:47.785061 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.782359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:37175 (local address 127.29.142.131:43683)
0806 06:35:47.782424 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.782426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.782436 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:35:47.782542 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.782543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.782552 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.782622 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:47.782811 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.783280 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.784317 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.784685 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.784776 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.784785 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.784942 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.784981 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:35:47.785061 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.782131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:43579 (local address 127.29.142.129:44773)
0806 06:35:47.782225 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.782228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.782301 (+ 73us) server_negotiation.cc:407] Connection header received
0806 06:35:47.782430 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.782433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.782444 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.782529 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:47.782741 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.783243 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.784309 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.784677 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.784775 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.784784 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.784947 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.784987 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 06:35:47.785089 1101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.782023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:44773 (local address 127.29.142.130:43579)
0806 06:35:47.782180 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.782198 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.782303 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.782535 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.782535 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.782538 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:47.782637 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.782639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.783351 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.783352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.784197 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.784200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.784775 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.784776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.784856 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.784862 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.784939 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.784987 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260806 06:35:47.785125 1102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.782257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:43683 (local address 127.29.142.130:37175)
0806 06:35:47.782370 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.782381 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.782446 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.782629 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.782630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.782633 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:47.782726 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.782728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.783403 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.783404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.784201 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.784203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.784776 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.784778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.784856 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.784860 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.784935 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.784979 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":48,"thread_start_us":79,"threads_started":1}
I20260806 06:35:47.785609 970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2277dec49cb7408b81b4fb22934d314c" candidate_uuid: "8ea4545546aa4df2b96ddaeed3243847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1854ee34f734988be91601bb216ca40" is_pre_election: true
I20260806 06:35:47.785658 579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2277dec49cb7408b81b4fb22934d314c" candidate_uuid: "8ea4545546aa4df2b96ddaeed3243847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f167c29d7bc4c1faaf2696c72e35658" is_pre_election: true
I20260806 06:35:47.785729 970 raft_consensus.cc:2454] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ea4545546aa4df2b96ddaeed3243847 in term 0.
I20260806 06:35:47.785789 579 raft_consensus.cc:2454] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ea4545546aa4df2b96ddaeed3243847 in term 0.
I20260806 06:35:47.785949 678 leader_election.cc:302] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea4545546aa4df2b96ddaeed3243847, b1854ee34f734988be91601bb216ca40; no voters:
I20260806 06:35:47.786144 1083 raft_consensus.cc:2781] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:35:47.786195 1083 raft_consensus.cc:482] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:35:47.786211 1083 raft_consensus.cc:3037] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:47.787020 1083 raft_consensus.cc:504] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.787173 1083 leader_election.cc:288] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [CANDIDATE]: Term 1 election: Requested vote from peers 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129:44773), b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683)
I20260806 06:35:47.787406 970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2277dec49cb7408b81b4fb22934d314c" candidate_uuid: "8ea4545546aa4df2b96ddaeed3243847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1854ee34f734988be91601bb216ca40"
I20260806 06:35:47.787472 970 raft_consensus.cc:3037] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:47.787408 579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2277dec49cb7408b81b4fb22934d314c" candidate_uuid: "8ea4545546aa4df2b96ddaeed3243847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f167c29d7bc4c1faaf2696c72e35658"
I20260806 06:35:47.787513 579 raft_consensus.cc:3037] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:47.788123 970 raft_consensus.cc:2454] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea4545546aa4df2b96ddaeed3243847 in term 1.
I20260806 06:35:47.788264 579 raft_consensus.cc:2454] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea4545546aa4df2b96ddaeed3243847 in term 1.
I20260806 06:35:47.788309 678 leader_election.cc:302] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea4545546aa4df2b96ddaeed3243847, b1854ee34f734988be91601bb216ca40; no voters:
I20260806 06:35:47.788389 1083 raft_consensus.cc:2781] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:47.788540 1083 raft_consensus.cc:686] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [term 1 LEADER]: Becoming Leader. State: Replica: 8ea4545546aa4df2b96ddaeed3243847, State: Running, Role: LEADER
I20260806 06:35:47.788671 1083 consensus_queue.cc:237] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.789201 1103 tablet_replica.cc:439] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: TxnStatusTablet state changed. Reason: New leader 8ea4545546aa4df2b96ddaeed3243847. Latest consensus state: current_term: 1 leader_uuid: "8ea4545546aa4df2b96ddaeed3243847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } }
I20260806 06:35:47.789296 1103 tablet_replica.cc:442] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:35:47.789486 1106 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:35:47.789623 1106 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:35:47.789709 349 catalog_manager.cc:5495] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130). New cstate: current_term: 1 leader_uuid: "8ea4545546aa4df2b96ddaeed3243847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } health_report { overall_health: UNKNOWN } } }
I20260806 06:35:47.813190 1070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.810440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55244 (local address 127.29.142.130:41743)
0806 06:35:47.810516 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.810518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.810544 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:35:47.810714 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.810715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.810737 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.810814 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:47.810971 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.811476 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.812268 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.812493 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.812580 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.812792 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:47.812921 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:47.813029 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.813074 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.813099 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:35:47.813190 1096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.810372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:41743 (local address 127.0.0.1:55244)
0806 06:35:47.810490 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.810504 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.810567 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.810828 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.810830 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.810831 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:35:47.810892 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.810894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.811574 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.811575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.812166 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.812168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.812582 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.812583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.812658 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.812675 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:35:47.813002 (+ 327us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:35:47.813014 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.813066 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.813118 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 06:35:47.815253 970 raft_consensus.cc:1261] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Refusing update from remote peer 8ea4545546aa4df2b96ddaeed3243847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:47.815342 579 raft_consensus.cc:1261] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Refusing update from remote peer 8ea4545546aa4df2b96ddaeed3243847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:47.815611 1103 consensus_queue.cc:1035] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:47.815768 1083 consensus_queue.cc:1035] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:47.817123 1084 tablet_replica.cc:439] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40: TxnStatusTablet state changed. Reason: New leader 8ea4545546aa4df2b96ddaeed3243847. Latest consensus state: current_term: 1 leader_uuid: "8ea4545546aa4df2b96ddaeed3243847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } }
I20260806 06:35:47.817230 1084 tablet_replica.cc:442] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:47.817350 1085 tablet_replica.cc:439] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658: TxnStatusTablet state changed. Reason: New leader 8ea4545546aa4df2b96ddaeed3243847. Latest consensus state: current_term: 1 leader_uuid: "8ea4545546aa4df2b96ddaeed3243847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } }
I20260806 06:35:47.817469 1085 tablet_replica.cc:442] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:47.819869 1103 tablet_replica.cc:439] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ea4545546aa4df2b96ddaeed3243847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } }
I20260806 06:35:47.819990 1103 tablet_replica.cc:442] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:35:47.819998 1084 tablet_replica.cc:439] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ea4545546aa4df2b96ddaeed3243847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } }
I20260806 06:35:47.820071 1084 tablet_replica.cc:442] T 2277dec49cb7408b81b4fb22934d314c P b1854ee34f734988be91601bb216ca40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:47.820562 1083 tablet_replica.cc:439] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ea4545546aa4df2b96ddaeed3243847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } }
I20260806 06:35:47.820652 1083 tablet_replica.cc:442] T 2277dec49cb7408b81b4fb22934d314c P 8ea4545546aa4df2b96ddaeed3243847: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:35:47.820876 1110 mvcc.cc:205] Tried to move back new op lower bound from 7315448413448384512 to 7315448413343105024. Current Snapshot: MvccSnapshot[applied={T|T < 7315448413448384512 or (T in {7315448413448384512})}]
I20260806 06:35:47.820977 1114 mvcc.cc:205] Tried to move back new op lower bound from 7315448413448384512 to 7315448413343105024. Current Snapshot: MvccSnapshot[applied={T|T < 7315448413448384512}]
I20260806 06:35:47.821612 1085 tablet_replica.cc:439] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ea4545546aa4df2b96ddaeed3243847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } } }
I20260806 06:35:47.821719 1085 tablet_replica.cc:442] T 2277dec49cb7408b81b4fb22934d314c P 8f167c29d7bc4c1faaf2696c72e35658: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:35:47.821882 1113 mvcc.cc:205] Tried to move back new op lower bound from 7315448413448384512 to 7315448413343105024. Current Snapshot: MvccSnapshot[applied={T|T < 7315448413448384512}]
I20260806 06:35:47.825750 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.823112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58402 (local address 127.29.142.129:44773)
0806 06:35:47.823160 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.823162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.823172 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:35:47.823252 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.823253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.823276 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.823353 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:47.823528 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.824017 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.824901 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.825103 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.825188 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.825364 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:47.825529 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:47.825602 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.825643 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.825667 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:35:47.832026 1128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.828899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:41743 (local address 127.29.142.129:54323)
0806 06:35:47.829091 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.829112 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.829173 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.829317 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.829318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.829319 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:35:47.829378 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.829379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.830284 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.830285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.831053 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.831056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.831559 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.831560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.831618 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.831635 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:35:47.831863 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:35:47.831865 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.831904 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.831945 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":115,"threads_started":1}
I20260806 06:35:47.832019 1070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.828970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:54323 (local address 127.29.142.130:41743)
0806 06:35:47.829018 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.829020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.829179 (+ 159us) server_negotiation.cc:407] Connection header received
0806 06:35:47.829216 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.829217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.829231 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.829297 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:47.829446 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.830185 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.831140 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.831458 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.831536 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.831761 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:47.831797 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:47.831853 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.831908 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.831931 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:35:47.835084 970 raft_consensus.cc:1261] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Refusing update from remote peer 8f167c29d7bc4c1faaf2696c72e35658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:47.835197 778 raft_consensus.cc:1261] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Refusing update from remote peer 8f167c29d7bc4c1faaf2696c72e35658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:47.835577 1089 consensus_queue.cc:1035] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:47.835772 1085 consensus_queue.cc:1035] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:47.845937 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.842272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:44773 (local address 127.29.142.130:45709)
0806 06:35:47.842799 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:47.842811 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:47.842874 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:47.843005 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:47.843006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:47.843007 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:35:47.843077 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.843079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.844014 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.844015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.844980 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:47.844983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.845406 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.845408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:47.845473 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.845491 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:35:47.845719 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:35:47.845721 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:35:47.845766 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:35:47.845817 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":131,"threads_started":1}
I20260806 06:35:47.845983 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:47.842369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:45709 (local address 127.29.142.129:44773)
0806 06:35:47.842437 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:47.842439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:47.842875 (+ 436us) server_negotiation.cc:407] Connection header received
0806 06:35:47.842923 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:47.842924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:47.842935 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:47.842998 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:47.843139 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.843909 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.845094 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:47.845303 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:47.845363 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:47.845606 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:47.845644 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:47.845700 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:47.845863 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 06:35:47.845894 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:35:47.854380 349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:35:47.854521 349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:35:47.855474 349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e: Sending DeleteTablet for 3 replicas of tablet 59e4741c155344f789fc16ffb10e5a5e
I20260806 06:35:47.855815 942 tablet_service.cc:1514] Processing DeleteTablet for tablet 59e4741c155344f789fc16ffb10e5a5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:47 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:58002
I20260806 06:35:47.856182 551 tablet_service.cc:1514] Processing DeleteTablet for tablet 59e4741c155344f789fc16ffb10e5a5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:47 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:58400
I20260806 06:35:47.856148 349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:35:47.856297 349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:35:47.856488 753 tablet_service.cc:1514] Processing DeleteTablet for tablet 59e4741c155344f789fc16ffb10e5a5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:47 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:55240
I20260806 06:35:47.856748 1146 tablet_replica.cc:330] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658: stopping tablet replica
I20260806 06:35:47.856856 1146 raft_consensus.cc:2229] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:35:47.856964 1146 raft_consensus.cc:2258] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:47.857132 1147 tablet_replica.cc:330] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847: stopping tablet replica
I20260806 06:35:47.857236 1147 raft_consensus.cc:2229] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:35:47.857293 1147 raft_consensus.cc:2258] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:47.857905 551 tablet_service.cc:1467] Processing CreateTablet for tablet 9e400d19f23f440a8028469b6860da8b (DEFAULT_TABLE table=test-table [id=1bbe8e4b4681489fa2bf3279da7c998e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:47.858042 551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e400d19f23f440a8028469b6860da8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:47.858127 753 tablet_service.cc:1467] Processing CreateTablet for tablet 9e400d19f23f440a8028469b6860da8b (DEFAULT_TABLE table=test-table [id=1bbe8e4b4681489fa2bf3279da7c998e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:47.858253 753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e400d19f23f440a8028469b6860da8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:47.859571 1075 tablet_bootstrap.cc:492] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658: Bootstrap starting.
I20260806 06:35:47.859612 1079 tablet_bootstrap.cc:492] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847: Bootstrap starting.
I20260806 06:35:47.860014 1079 tablet_bootstrap.cc:654] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:47.860090 1075 tablet_bootstrap.cc:654] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:47.860443 942 tablet_service.cc:1467] Processing CreateTablet for tablet 9e400d19f23f440a8028469b6860da8b (DEFAULT_TABLE table=test-table [id=1bbe8e4b4681489fa2bf3279da7c998e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:47.860558 942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e400d19f23f440a8028469b6860da8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:47.860720 1079 tablet_bootstrap.cc:492] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847: No bootstrap required, opened a new log
I20260806 06:35:47.860770 1079 ts_tablet_manager.cc:1397] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:35:47.860913 1079 raft_consensus.cc:348] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.860981 1079 raft_consensus.cc:374] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:47.861002 1079 raft_consensus.cc:729] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea4545546aa4df2b96ddaeed3243847, State: Initialized, Role: FOLLOWER
I20260806 06:35:47.861047 1079 consensus_queue.cc:260] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.861132 1079 ts_tablet_manager.cc:1428] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:35:47.861948 1077 tablet_bootstrap.cc:492] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40: Bootstrap starting.
I20260806 06:35:47.862375 1077 tablet_bootstrap.cc:654] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:47.863246 1077 tablet_bootstrap.cc:492] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40: No bootstrap required, opened a new log
I20260806 06:35:47.863317 1077 ts_tablet_manager.cc:1397] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:35:47.863453 1077 raft_consensus.cc:348] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.863544 1077 raft_consensus.cc:374] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:47.863567 1077 raft_consensus.cc:729] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1854ee34f734988be91601bb216ca40, State: Initialized, Role: FOLLOWER
I20260806 06:35:47.863617 1077 consensus_queue.cc:260] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.863718 1077 ts_tablet_manager.cc:1428] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:35:47.863862 1147 ts_tablet_manager.cc:1905] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:47.864310 1146 ts_tablet_manager.cc:1905] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:47.866161 1147 ts_tablet_manager.cc:1918] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:35:47.866247 1147 log.cc:1196] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/59e4741c155344f789fc16ffb10e5a5e
I20260806 06:35:47.866575 1147 ts_tablet_manager.cc:1939] T 59e4741c155344f789fc16ffb10e5a5e P 8ea4545546aa4df2b96ddaeed3243847: Deleting consensus metadata
I20260806 06:35:47.866732 1145 tablet_replica.cc:330] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40: stopping tablet replica
I20260806 06:35:47.866878 1145 raft_consensus.cc:2229] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:35:47.866955 1145 raft_consensus.cc:2258] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:47.867132 319 catalog_manager.cc:4841] TS 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130:41743): tablet 59e4741c155344f789fc16ffb10e5a5e (table test-table [id=46d12391c369428dbe19e9e83ac224be]) successfully deleted
I20260806 06:35:47.867436 1145 ts_tablet_manager.cc:1905] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:47.867630 1075 tablet_bootstrap.cc:492] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658: No bootstrap required, opened a new log
I20260806 06:35:47.867676 1075 ts_tablet_manager.cc:1397] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260806 06:35:47.867806 1075 raft_consensus.cc:348] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.867872 1075 raft_consensus.cc:374] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:47.867887 1075 raft_consensus.cc:729] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f167c29d7bc4c1faaf2696c72e35658, State: Initialized, Role: FOLLOWER
I20260806 06:35:47.867977 1075 consensus_queue.cc:260] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:47.868089 1075 ts_tablet_manager.cc:1428] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:35:47.868391 1146 ts_tablet_manager.cc:1918] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:35:47.868470 1146 log.cc:1196] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/wal/wals/59e4741c155344f789fc16ffb10e5a5e
I20260806 06:35:47.868732 1146 ts_tablet_manager.cc:1939] T 59e4741c155344f789fc16ffb10e5a5e P 8f167c29d7bc4c1faaf2696c72e35658: Deleting consensus metadata
I20260806 06:35:47.869091 1145 ts_tablet_manager.cc:1918] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:35:47.869163 1145 log.cc:1196] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/59e4741c155344f789fc16ffb10e5a5e
I20260806 06:35:47.869233 323 catalog_manager.cc:4841] TS 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129:44773): tablet 59e4741c155344f789fc16ffb10e5a5e (table test-table [id=46d12391c369428dbe19e9e83ac224be]) successfully deleted
I20260806 06:35:47.869423 1145 ts_tablet_manager.cc:1939] T 59e4741c155344f789fc16ffb10e5a5e P b1854ee34f734988be91601bb216ca40: Deleting consensus metadata
I20260806 06:35:47.869853 323 catalog_manager.cc:4841] TS b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683): tablet 59e4741c155344f789fc16ffb10e5a5e (table test-table [id=46d12391c369428dbe19e9e83ac224be]) successfully deleted
I20260806 06:35:48.021850 1085 raft_consensus.cc:482] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:35:48.022027 1085 raft_consensus.cc:504] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:48.022279 1085 leader_election.cc:288] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130:41743), b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683)
I20260806 06:35:48.022578 970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e400d19f23f440a8028469b6860da8b" candidate_uuid: "8f167c29d7bc4c1faaf2696c72e35658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1854ee34f734988be91601bb216ca40" is_pre_election: true
I20260806 06:35:48.022578 778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e400d19f23f440a8028469b6860da8b" candidate_uuid: "8f167c29d7bc4c1faaf2696c72e35658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea4545546aa4df2b96ddaeed3243847" is_pre_election: true
I20260806 06:35:48.022745 970 raft_consensus.cc:2454] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f167c29d7bc4c1faaf2696c72e35658 in term 0.
I20260806 06:35:48.022745 778 raft_consensus.cc:2454] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f167c29d7bc4c1faaf2696c72e35658 in term 0.
I20260806 06:35:48.022967 484 leader_election.cc:302] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f167c29d7bc4c1faaf2696c72e35658, b1854ee34f734988be91601bb216ca40; no voters:
I20260806 06:35:48.023109 1085 raft_consensus.cc:2781] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:35:48.023155 1085 raft_consensus.cc:482] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:35:48.023185 1085 raft_consensus.cc:3037] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:48.023962 1085 raft_consensus.cc:504] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:48.024129 1085 leader_election.cc:288] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [CANDIDATE]: Term 1 election: Requested vote from peers 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130:41743), b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683)
I20260806 06:35:48.024335 970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e400d19f23f440a8028469b6860da8b" candidate_uuid: "8f167c29d7bc4c1faaf2696c72e35658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1854ee34f734988be91601bb216ca40"
I20260806 06:35:48.024335 778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e400d19f23f440a8028469b6860da8b" candidate_uuid: "8f167c29d7bc4c1faaf2696c72e35658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea4545546aa4df2b96ddaeed3243847"
I20260806 06:35:48.024430 778 raft_consensus.cc:3037] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:48.024428 970 raft_consensus.cc:3037] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:48.025277 970 raft_consensus.cc:2454] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f167c29d7bc4c1faaf2696c72e35658 in term 1.
I20260806 06:35:48.025277 778 raft_consensus.cc:2454] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f167c29d7bc4c1faaf2696c72e35658 in term 1.
I20260806 06:35:48.025482 480 leader_election.cc:302] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea4545546aa4df2b96ddaeed3243847, 8f167c29d7bc4c1faaf2696c72e35658; no voters:
I20260806 06:35:48.025597 1085 raft_consensus.cc:2781] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:48.025663 1085 raft_consensus.cc:686] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 LEADER]: Becoming Leader. State: Replica: 8f167c29d7bc4c1faaf2696c72e35658, State: Running, Role: LEADER
I20260806 06:35:48.025713 1085 consensus_queue.cc:237] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:48.026371 349 catalog_manager.cc:5495] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129). New cstate: current_term: 1 leader_uuid: "8f167c29d7bc4c1faaf2696c72e35658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } health_report { overall_health: UNKNOWN } } }
I20260806 06:35:48.048863 968 raft_consensus.cc:1261] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Refusing update from remote peer 8f167c29d7bc4c1faaf2696c72e35658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:48.048916 778 raft_consensus.cc:1261] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Refusing update from remote peer 8f167c29d7bc4c1faaf2696c72e35658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:48.049149 1139 consensus_queue.cc:1035] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:48.049312 1085 consensus_queue.cc:1035] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:35:48.054772 1113 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 06:35:49.576335 348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:35:49.576457 348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:35:49.577433 348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e: Sending DeleteTablet for 3 replicas of tablet 9e400d19f23f440a8028469b6860da8b
I20260806 06:35:49.577785 942 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e400d19f23f440a8028469b6860da8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:49 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:58002
I20260806 06:35:49.577852 753 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e400d19f23f440a8028469b6860da8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:49 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:55240
I20260806 06:35:49.577808 551 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e400d19f23f440a8028469b6860da8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:49 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:58400
I20260806 06:35:49.578050 1193 tablet_replica.cc:330] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40: stopping tablet replica
I20260806 06:35:49.578050 1194 tablet_replica.cc:330] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847: stopping tablet replica
I20260806 06:35:49.578094 1195 tablet_replica.cc:330] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658: stopping tablet replica
I20260806 06:35:49.578141 1194 raft_consensus.cc:2229] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:35:49.578188 1195 raft_consensus.cc:2229] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:35:49.578243 1194 raft_consensus.cc:2258] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:49.578336 1195 raft_consensus.cc:2258] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:49.578354 1193 raft_consensus.cc:2229] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:35:49.578588 1193 raft_consensus.cc:2258] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:49.578586 1194 ts_tablet_manager.cc:1905] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:49.578625 1195 ts_tablet_manager.cc:1905] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:49.580855 1194 ts_tablet_manager.cc:1918] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:35:49.580984 1194 log.cc:1196] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/9e400d19f23f440a8028469b6860da8b
I20260806 06:35:49.581274 1194 ts_tablet_manager.cc:1939] T 9e400d19f23f440a8028469b6860da8b P 8ea4545546aa4df2b96ddaeed3243847: Deleting consensus metadata
I20260806 06:35:49.581250 1195 ts_tablet_manager.cc:1918] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:35:49.581327 1195 log.cc:1196] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/wal/wals/9e400d19f23f440a8028469b6860da8b
I20260806 06:35:49.581729 1195 ts_tablet_manager.cc:1939] T 9e400d19f23f440a8028469b6860da8b P 8f167c29d7bc4c1faaf2696c72e35658: Deleting consensus metadata
I20260806 06:35:49.582154 1193 ts_tablet_manager.cc:1905] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:49.582264 323 catalog_manager.cc:4841] TS 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129:44773): tablet 9e400d19f23f440a8028469b6860da8b (table test-table [id=1bbe8e4b4681489fa2bf3279da7c998e]) successfully deleted
I20260806 06:35:49.582803 319 catalog_manager.cc:4841] TS 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130:41743): tablet 9e400d19f23f440a8028469b6860da8b (table test-table [id=1bbe8e4b4681489fa2bf3279da7c998e]) successfully deleted
I20260806 06:35:49.583918 1193 ts_tablet_manager.cc:1918] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:35:49.583993 1193 log.cc:1196] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/9e400d19f23f440a8028469b6860da8b
I20260806 06:35:49.584271 1193 ts_tablet_manager.cc:1939] T 9e400d19f23f440a8028469b6860da8b P b1854ee34f734988be91601bb216ca40: Deleting consensus metadata
I20260806 06:35:49.584661 323 catalog_manager.cc:4841] TS b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683): tablet 9e400d19f23f440a8028469b6860da8b (table test-table [id=1bbe8e4b4681489fa2bf3279da7c998e]) successfully deleted
Aug 06 06:35:49 dist-test-slave-3kk6 krb5kdc[32749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998149, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:35:49 dist-test-slave-3kk6 krb5kdc[32749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998149, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:49.608925 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:49.599801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51934 (local address 127.29.142.190:41341)
0806 06:35:49.600003 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:49.600005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:49.600077 (+ 72us) server_negotiation.cc:407] Connection header received
0806 06:35:49.600142 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:49.600143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:49.600157 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:49.600244 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:49.600908 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:49.601457 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:49.602384 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:49.602594 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:49.602780 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:49.605660 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:49.605663 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:49.605663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:49.605673 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:49.607546 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:49.607977 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:49.607978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:49.607979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:49.608059 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:49.608297 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:49.608298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:49.608298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:49.608436 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:49.608553 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:49.608697 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 06:35:49.608801 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":89,"threads_started":1}
I20260806 06:35:49.611064 348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:35:49.611369 348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:35:49.613004 551 tablet_service.cc:1467] Processing CreateTablet for tablet bd5ed994fc514796ab1c062921fc1e91 (DEFAULT_TABLE table=test-table [id=026acfd8d22945bf96cb64a3af37fda5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:49.613032 753 tablet_service.cc:1467] Processing CreateTablet for tablet bd5ed994fc514796ab1c062921fc1e91 (DEFAULT_TABLE table=test-table [id=026acfd8d22945bf96cb64a3af37fda5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:49.613054 942 tablet_service.cc:1467] Processing CreateTablet for tablet bd5ed994fc514796ab1c062921fc1e91 (DEFAULT_TABLE table=test-table [id=026acfd8d22945bf96cb64a3af37fda5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:49.613193 942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5ed994fc514796ab1c062921fc1e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:49.613194 753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5ed994fc514796ab1c062921fc1e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:49.613204 551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd5ed994fc514796ab1c062921fc1e91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:49.614338 1207 tablet_bootstrap.cc:492] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658: Bootstrap starting.
I20260806 06:35:49.614777 1209 tablet_bootstrap.cc:492] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40: Bootstrap starting.
I20260806 06:35:49.615121 1207 tablet_bootstrap.cc:654] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:49.615221 1209 tablet_bootstrap.cc:654] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:49.615566 1211 tablet_bootstrap.cc:492] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847: Bootstrap starting.
I20260806 06:35:49.615876 1207 tablet_bootstrap.cc:492] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658: No bootstrap required, opened a new log
I20260806 06:35:49.615938 1207 ts_tablet_manager.cc:1397] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:35:49.616030 1211 tablet_bootstrap.cc:654] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:49.616101 1207 raft_consensus.cc:348] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:49.616199 1207 raft_consensus.cc:374] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:49.616222 1207 raft_consensus.cc:729] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f167c29d7bc4c1faaf2696c72e35658, State: Initialized, Role: FOLLOWER
I20260806 06:35:49.616302 1207 consensus_queue.cc:260] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:49.616433 1207 ts_tablet_manager.cc:1428] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:35:49.616722 1209 tablet_bootstrap.cc:492] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40: No bootstrap required, opened a new log
I20260806 06:35:49.616726 1211 tablet_bootstrap.cc:492] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847: No bootstrap required, opened a new log
I20260806 06:35:49.616781 1209 ts_tablet_manager.cc:1397] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:35:49.616781 1211 ts_tablet_manager.cc:1397] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:49.616902 1211 raft_consensus.cc:348] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:49.616982 1209 raft_consensus.cc:348] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:49.617043 1211 raft_consensus.cc:374] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:49.617100 1211 raft_consensus.cc:729] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea4545546aa4df2b96ddaeed3243847, State: Initialized, Role: FOLLOWER
I20260806 06:35:49.617105 1209 raft_consensus.cc:374] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:49.617136 1209 raft_consensus.cc:729] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1854ee34f734988be91601bb216ca40, State: Initialized, Role: FOLLOWER
I20260806 06:35:49.617231 1211 consensus_queue.cc:260] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:49.617246 1209 consensus_queue.cc:260] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:49.617427 1209 ts_tablet_manager.cc:1428] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:35:49.617425 1211 ts_tablet_manager.cc:1428] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:49.930356 1163 raft_consensus.cc:482] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:35:49.930497 1163 raft_consensus.cc:504] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:49.930672 1163 leader_election.cc:288] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129:44773), b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683)
I20260806 06:35:49.931077 968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5ed994fc514796ab1c062921fc1e91" candidate_uuid: "8ea4545546aa4df2b96ddaeed3243847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1854ee34f734988be91601bb216ca40" is_pre_election: true
I20260806 06:35:49.931077 579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5ed994fc514796ab1c062921fc1e91" candidate_uuid: "8ea4545546aa4df2b96ddaeed3243847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f167c29d7bc4c1faaf2696c72e35658" is_pre_election: true
I20260806 06:35:49.931218 968 raft_consensus.cc:2454] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ea4545546aa4df2b96ddaeed3243847 in term 0.
I20260806 06:35:49.931218 579 raft_consensus.cc:2454] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ea4545546aa4df2b96ddaeed3243847 in term 0.
I20260806 06:35:49.931468 678 leader_election.cc:302] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea4545546aa4df2b96ddaeed3243847, 8f167c29d7bc4c1faaf2696c72e35658; no voters:
I20260806 06:35:49.931602 1163 raft_consensus.cc:2781] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:35:49.931661 1163 raft_consensus.cc:482] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:35:49.931684 1163 raft_consensus.cc:3037] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:49.932484 1163 raft_consensus.cc:504] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:49.932639 1163 leader_election.cc:288] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [CANDIDATE]: Term 1 election: Requested vote from peers 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129:44773), b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683)
I20260806 06:35:49.932943 968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5ed994fc514796ab1c062921fc1e91" candidate_uuid: "8ea4545546aa4df2b96ddaeed3243847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1854ee34f734988be91601bb216ca40"
I20260806 06:35:49.932943 579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd5ed994fc514796ab1c062921fc1e91" candidate_uuid: "8ea4545546aa4df2b96ddaeed3243847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f167c29d7bc4c1faaf2696c72e35658"
I20260806 06:35:49.933053 968 raft_consensus.cc:3037] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:49.933053 579 raft_consensus.cc:3037] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:49.933995 968 raft_consensus.cc:2454] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea4545546aa4df2b96ddaeed3243847 in term 1.
I20260806 06:35:49.933995 579 raft_consensus.cc:2454] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea4545546aa4df2b96ddaeed3243847 in term 1.
I20260806 06:35:49.934254 678 leader_election.cc:302] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea4545546aa4df2b96ddaeed3243847, 8f167c29d7bc4c1faaf2696c72e35658; no voters:
I20260806 06:35:49.934361 1163 raft_consensus.cc:2781] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:49.934418 1163 raft_consensus.cc:686] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 1 LEADER]: Becoming Leader. State: Replica: 8ea4545546aa4df2b96ddaeed3243847, State: Running, Role: LEADER
I20260806 06:35:49.934482 1163 consensus_queue.cc:237] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:49.935352 349 catalog_manager.cc:5495] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130). New cstate: current_term: 1 leader_uuid: "8ea4545546aa4df2b96ddaeed3243847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } health_report { overall_health: UNKNOWN } } }
I20260806 06:35:49.978165 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:49.975479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.29.142.130:41743)
0806 06:35:49.975688 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:49.975689 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:49.975698 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:35:49.975720 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:49.975720 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:49.975731 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:49.975798 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:49.975993 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:49.976638 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:49.977432 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:49.977634 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:49.977690 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:49.977888 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:49.977921 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:49.977989 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:49.978026 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 06:35:49.978052 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":108,"threads_started":1}
I20260806 06:35:49.983727 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:49.981312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:60971 (local address 127.29.142.130:41743)
0806 06:35:49.981385 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:49.981387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:49.981402 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:35:49.981488 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:49.981489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:49.981498 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:49.981555 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:49.981691 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:49.982176 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:49.983064 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:49.983259 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:49.983319 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:49.983454 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:49.983482 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:49.983522 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:49.983603 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 06:35:49.983624 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:35:49.983738 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:49.981190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:41743 (local address 127.29.142.130:60971)
0806 06:35:49.981345 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:49.981359 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:49.981416 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:49.981572 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:49.981573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:49.981574 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:35:49.981621 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:49.981622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:49.982281 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:49.982282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:49.982977 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:49.982980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:49.983337 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:49.983337 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:49.983380 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:49.983384 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:35:49.983545 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:35:49.983546 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:35:49.983585 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 06:35:49.983625 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260806 06:35:49.986521 968 raft_consensus.cc:1261] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Refusing update from remote peer 8ea4545546aa4df2b96ddaeed3243847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:49.986543 579 raft_consensus.cc:1261] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Refusing update from remote peer 8ea4545546aa4df2b96ddaeed3243847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:49.986792 1163 consensus_queue.cc:1035] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:49.986979 1172 consensus_queue.cc:1035] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:49.999895 349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:35:50.000029 349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:35:50.000846 349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e: Sending DeleteTablet for 3 replicas of tablet bd5ed994fc514796ab1c062921fc1e91
I20260806 06:35:50.001362 942 tablet_service.cc:1514] Processing DeleteTablet for tablet bd5ed994fc514796ab1c062921fc1e91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:49 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:58002
I20260806 06:35:50.001390 551 tablet_service.cc:1514] Processing DeleteTablet for tablet bd5ed994fc514796ab1c062921fc1e91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:49 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:58400
I20260806 06:35:50.001507 1193 tablet_replica.cc:330] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40: stopping tablet replica
I20260806 06:35:50.001577 1193 raft_consensus.cc:2229] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:35:50.001636 1193 raft_consensus.cc:2258] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:50.001717 1195 tablet_replica.cc:330] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658: stopping tablet replica
I20260806 06:35:50.001780 1195 raft_consensus.cc:2229] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:35:50.001842 1195 raft_consensus.cc:2258] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:50.002416 349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:35:50.002601 349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:35:50.002923 751 tablet_service.cc:1514] Processing DeleteTablet for tablet bd5ed994fc514796ab1c062921fc1e91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:49 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:55240
I20260806 06:35:50.003026 1194 tablet_replica.cc:330] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847: stopping tablet replica
I20260806 06:35:50.003079 1194 raft_consensus.cc:2229] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:35:50.003186 1194 raft_consensus.cc:2258] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:50.003791 1194 ts_tablet_manager.cc:1905] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:50.005590 1194 ts_tablet_manager.cc:1918] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:35:50.005669 1194 log.cc:1196] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/bd5ed994fc514796ab1c062921fc1e91
I20260806 06:35:50.005903 942 tablet_service.cc:1467] Processing CreateTablet for tablet 3c2e3c8bab654e0489c1892b83e15d38 (DEFAULT_TABLE table=test-table [id=44e01cf041b942239f4feb27b0b73af0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:50.005963 1194 ts_tablet_manager.cc:1939] T bd5ed994fc514796ab1c062921fc1e91 P 8ea4545546aa4df2b96ddaeed3243847: Deleting consensus metadata
I20260806 06:35:50.006065 942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c2e3c8bab654e0489c1892b83e15d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:50.006042 551 tablet_service.cc:1467] Processing CreateTablet for tablet 3c2e3c8bab654e0489c1892b83e15d38 (DEFAULT_TABLE table=test-table [id=44e01cf041b942239f4feb27b0b73af0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:50.006207 551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c2e3c8bab654e0489c1892b83e15d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:50.007625 1207 tablet_bootstrap.cc:492] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658: Bootstrap starting.
I20260806 06:35:50.007757 1209 tablet_bootstrap.cc:492] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40: Bootstrap starting.
I20260806 06:35:50.008054 1207 tablet_bootstrap.cc:654] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:50.008117 319 catalog_manager.cc:4841] TS 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130:41743): tablet bd5ed994fc514796ab1c062921fc1e91 (table test-table [id=026acfd8d22945bf96cb64a3af37fda5]) successfully deleted
I20260806 06:35:50.008198 751 tablet_service.cc:1467] Processing CreateTablet for tablet 3c2e3c8bab654e0489c1892b83e15d38 (DEFAULT_TABLE table=test-table [id=44e01cf041b942239f4feb27b0b73af0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:35:50.008314 751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c2e3c8bab654e0489c1892b83e15d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:50.008591 1209 tablet_bootstrap.cc:654] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:50.009450 1209 tablet_bootstrap.cc:492] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40: No bootstrap required, opened a new log
I20260806 06:35:50.009511 1209 ts_tablet_manager.cc:1397] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:35:50.009743 1211 tablet_bootstrap.cc:492] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847: Bootstrap starting.
I20260806 06:35:50.009723 1209 raft_consensus.cc:348] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:50.009800 1209 raft_consensus.cc:374] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:50.009821 1209 raft_consensus.cc:729] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1854ee34f734988be91601bb216ca40, State: Initialized, Role: FOLLOWER
I20260806 06:35:50.009965 1209 consensus_queue.cc:260] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:50.010164 1211 tablet_bootstrap.cc:654] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:50.010203 1195 ts_tablet_manager.cc:1905] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:50.010586 1209 ts_tablet_manager.cc:1428] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:50.011488 1211 tablet_bootstrap.cc:492] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847: No bootstrap required, opened a new log
I20260806 06:35:50.011554 1211 ts_tablet_manager.cc:1397] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:35:50.011701 1211 raft_consensus.cc:348] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:50.011787 1211 raft_consensus.cc:374] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:50.011812 1211 raft_consensus.cc:729] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea4545546aa4df2b96ddaeed3243847, State: Initialized, Role: FOLLOWER
I20260806 06:35:50.011870 1211 consensus_queue.cc:260] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:50.011988 1211 ts_tablet_manager.cc:1428] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:35:50.012221 1195 ts_tablet_manager.cc:1918] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:35:50.012295 1195 log.cc:1196] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-0/wal/wals/bd5ed994fc514796ab1c062921fc1e91
I20260806 06:35:50.012614 1195 ts_tablet_manager.cc:1939] T bd5ed994fc514796ab1c062921fc1e91 P 8f167c29d7bc4c1faaf2696c72e35658: Deleting consensus metadata
I20260806 06:35:50.013062 323 catalog_manager.cc:4841] TS 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129:44773): tablet bd5ed994fc514796ab1c062921fc1e91 (table test-table [id=026acfd8d22945bf96cb64a3af37fda5]) successfully deleted
I20260806 06:35:50.013346 1193 ts_tablet_manager.cc:1905] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:50.013834 1207 tablet_bootstrap.cc:492] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658: No bootstrap required, opened a new log
I20260806 06:35:50.013896 1207 ts_tablet_manager.cc:1397] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260806 06:35:50.014017 1207 raft_consensus.cc:348] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:50.014094 1207 raft_consensus.cc:374] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:50.014114 1207 raft_consensus.cc:729] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f167c29d7bc4c1faaf2696c72e35658, State: Initialized, Role: FOLLOWER
I20260806 06:35:50.014160 1207 consensus_queue.cc:260] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:50.014249 1207 ts_tablet_manager.cc:1428] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:50.015242 1193 ts_tablet_manager.cc:1918] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:35:50.015348 1193 log.cc:1196] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/bd5ed994fc514796ab1c062921fc1e91
I20260806 06:35:50.015581 1193 ts_tablet_manager.cc:1939] T bd5ed994fc514796ab1c062921fc1e91 P b1854ee34f734988be91601bb216ca40: Deleting consensus metadata
I20260806 06:35:50.015981 323 catalog_manager.cc:4841] TS b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683): tablet bd5ed994fc514796ab1c062921fc1e91 (table test-table [id=026acfd8d22945bf96cb64a3af37fda5]) successfully deleted
I20260806 06:35:50.049595 1218 raft_consensus.cc:482] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:35:50.049744 1218 raft_consensus.cc:504] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:50.050149 1218 leader_election.cc:288] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129:44773), 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130:41743)
I20260806 06:35:50.054394 1244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:50.050380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:41743 (local address 127.29.142.131:33873)
0806 06:35:50.050762 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:50.050773 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:50.050849 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:50.051057 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:50.051058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:50.051059 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:50.051119 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:50.051120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.051934 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.051934 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:50.053282 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:50.053286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.054076 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.054077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:50.054144 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:50.054148 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:35:50.054234 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:35:50.054283 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":32,"thread_start_us":76,"threads_started":1}
I20260806 06:35:50.054426 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:50.050531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:33873 (local address 127.29.142.130:41743)
0806 06:35:50.050585 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:50.050587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:50.050823 (+ 236us) server_negotiation.cc:407] Connection header received
0806 06:35:50.050947 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:50.050948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:50.050959 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:50.051010 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:50.051248 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.051835 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.053397 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.053978 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.054070 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:50.054081 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:50.054243 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:35:50.054349 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:35:50.054760 778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2e3c8bab654e0489c1892b83e15d38" candidate_uuid: "b1854ee34f734988be91601bb216ca40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea4545546aa4df2b96ddaeed3243847" is_pre_election: true
I20260806 06:35:50.054970 778 raft_consensus.cc:2454] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1854ee34f734988be91601bb216ca40 in term 0.
I20260806 06:35:50.055205 877 leader_election.cc:302] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea4545546aa4df2b96ddaeed3243847, b1854ee34f734988be91601bb216ca40; no voters:
I20260806 06:35:50.055280 1243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:50.050380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:44773 (local address 127.29.142.131:48231)
0806 06:35:50.050637 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:50.050670 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:50.050743 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:50.050972 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:50.050973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:50.050976 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:35:50.051066 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:50.051068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.052172 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.052173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:50.053284 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:50.053286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.054938 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.054939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:50.055025 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:50.055029 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:35:50.055078 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:35:50.055124 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":1,"thread_start_us":113,"threads_started":1}
I20260806 06:35:50.055382 1218 raft_consensus.cc:2781] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:35:50.055420 1218 raft_consensus.cc:482] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:35:50.055440 1218 raft_consensus.cc:3037] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:50.055562 1245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:50.050511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:48231 (local address 127.29.142.129:44773)
0806 06:35:50.050643 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:50.050663 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:50.050747 (+ 84us) server_negotiation.cc:407] Connection header received
0806 06:35:50.050864 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:50.050865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:50.050877 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:50.050968 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:35:50.051150 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.052044 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.054122 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.054815 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.054891 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:50.054899 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:50.055392 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 06:35:50.055452 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
I20260806 06:35:50.056430 1218 raft_consensus.cc:504] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:50.056604 1218 leader_election.cc:288] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [CANDIDATE]: Term 1 election: Requested vote from peers 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129:44773), 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130:41743)
I20260806 06:35:50.056859 778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2e3c8bab654e0489c1892b83e15d38" candidate_uuid: "b1854ee34f734988be91601bb216ca40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea4545546aa4df2b96ddaeed3243847"
I20260806 06:35:50.056888 579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2e3c8bab654e0489c1892b83e15d38" candidate_uuid: "b1854ee34f734988be91601bb216ca40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f167c29d7bc4c1faaf2696c72e35658" is_pre_election: true
I20260806 06:35:50.056948 778 raft_consensus.cc:3037] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:50.056972 579 raft_consensus.cc:2454] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1854ee34f734988be91601bb216ca40 in term 0.
I20260806 06:35:50.056923 578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2e3c8bab654e0489c1892b83e15d38" candidate_uuid: "b1854ee34f734988be91601bb216ca40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f167c29d7bc4c1faaf2696c72e35658"
I20260806 06:35:50.057036 578 raft_consensus.cc:3037] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:50.057722 778 raft_consensus.cc:2454] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1854ee34f734988be91601bb216ca40 in term 1.
I20260806 06:35:50.057914 578 raft_consensus.cc:2454] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1854ee34f734988be91601bb216ca40 in term 1.
I20260806 06:35:50.058092 878 leader_election.cc:302] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f167c29d7bc4c1faaf2696c72e35658, b1854ee34f734988be91601bb216ca40; no voters:
I20260806 06:35:50.058223 1218 raft_consensus.cc:2781] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:50.058446 1218 raft_consensus.cc:686] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 1 LEADER]: Becoming Leader. State: Replica: b1854ee34f734988be91601bb216ca40, State: Running, Role: LEADER
I20260806 06:35:50.058538 1218 consensus_queue.cc:237] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } }
I20260806 06:35:50.059361 348 catalog_manager.cc:5495] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1854ee34f734988be91601bb216ca40 (127.29.142.131). New cstate: current_term: 1 leader_uuid: "b1854ee34f734988be91601bb216ca40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1854ee34f734988be91601bb216ca40" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 43683 } health_report { overall_health: HEALTHY } } }
I20260806 06:35:50.078410 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:50.075457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40612 (local address 127.29.142.131:43683)
0806 06:35:50.075642 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:50.075644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:50.075651 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:35:50.075705 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:50.075705 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:50.075748 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:50.075859 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:50.075977 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.076604 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.077387 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.077669 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.077764 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:50.077929 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:50.078058 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:50.078158 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:50.078200 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 06:35:50.078231 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":130,"threads_started":1}
I20260806 06:35:50.084730 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:50.081762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:42499 (local address 127.29.142.130:41743)
0806 06:35:50.081811 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:50.081813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:50.081953 (+ 140us) server_negotiation.cc:407] Connection header received
0806 06:35:50.081982 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:50.081983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:50.081995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:50.082052 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:50.082213 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.082952 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.083734 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.084030 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.084113 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:50.084393 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:50.084433 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:50.084500 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:50.084626 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 06:35:50.084651 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:35:50.085110 1256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:50.081591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:41743 (local address 127.29.142.131:42499)
0806 06:35:50.081834 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:50.081852 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:50.081910 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:50.082060 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:50.082061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:50.082062 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:35:50.082135 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:50.082136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.083039 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.083040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:50.083631 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:50.083633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.084181 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.084182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:50.084271 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:50.084290 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:35:50.084536 (+ 246us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:35:50.084540 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:35:50.084598 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:35:50.084643 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":99,"threads_started":1}
I20260806 06:35:50.087814 778 raft_consensus.cc:1261] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Refusing update from remote peer b1854ee34f734988be91601bb216ca40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:50.087889 578 raft_consensus.cc:1261] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Refusing update from remote peer b1854ee34f734988be91601bb216ca40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:35:50.088182 1246 consensus_queue.cc:1035] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea4545546aa4df2b96ddaeed3243847" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:50.088368 1218 consensus_queue.cc:1035] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f167c29d7bc4c1faaf2696c72e35658" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:35:50.097064 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:50.094098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:43683 (local address 127.29.142.130:48591)
0806 06:35:50.094175 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:50.094189 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:50.094241 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:50.094388 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:50.094389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:50.094391 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:35:50.094440 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:50.094442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.095371 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.095372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:50.096170 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:50.096173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.096663 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.096664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:50.096716 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:50.096722 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:35:50.096890 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:35:50.096891 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:35:50.096929 (+ 38us) client_negotiation.cc:240] Negotiation successful
0806 06:35:50.096977 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 06:35:50.097353 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:50.094127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:48591 (local address 127.29.142.131:43683)
0806 06:35:50.094180 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:50.094182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:50.094228 (+ 46us) server_negotiation.cc:407] Connection header received
0806 06:35:50.094300 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:50.094300 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:50.094321 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:50.094385 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:35:50.094500 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.095259 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.096273 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:50.096566 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:50.096625 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:50.096792 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:35:50.096826 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:35:50.096869 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:50.097251 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 06:35:50.097278 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 06:35:50.097918 1164 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 06:35:51.625869 348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:35:51.626047 348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:35:51.627027 348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f8a86b6af4b4dd5a790c825e478b47e: Sending DeleteTablet for 3 replicas of tablet 3c2e3c8bab654e0489c1892b83e15d38
I20260806 06:35:51.627327 551 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c2e3c8bab654e0489c1892b83e15d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:51 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:58400
I20260806 06:35:51.627351 751 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c2e3c8bab654e0489c1892b83e15d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:51 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:55240
I20260806 06:35:51.627449 942 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c2e3c8bab654e0489c1892b83e15d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:35:51 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:58002
I20260806 06:35:51.627609 1297 tablet_replica.cc:330] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847: stopping tablet replica
I20260806 06:35:51.627674 1297 raft_consensus.cc:2229] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:35:51.627719 1297 raft_consensus.cc:2258] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:51.627710 1292 tablet_replica.cc:330] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658: stopping tablet replica
I20260806 06:35:51.627775 1292 raft_consensus.cc:2229] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:35:51.627821 1292 raft_consensus.cc:2258] T 3c2e3c8bab654e0489c1892b83e15d38 P 8f167c29d7bc4c1faaf2696c72e35658 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:51.627830 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 459
I20260806 06:35:51.631718 1299 tablet_replica.cc:330] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40: stopping tablet replica
I20260806 06:35:51.631822 1299 raft_consensus.cc:2229] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:35:51.631911 1297 ts_tablet_manager.cc:1905] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:51.631948 1299 raft_consensus.cc:2258] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:35:51.634420 1299 ts_tablet_manager.cc:1905] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:35:51.634559 1297 ts_tablet_manager.cc:1918] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:35:51.634685 1297 log.cc:1196] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/3c2e3c8bab654e0489c1892b83e15d38
I20260806 06:35:51.634995 1297 ts_tablet_manager.cc:1939] T 3c2e3c8bab654e0489c1892b83e15d38 P 8ea4545546aa4df2b96ddaeed3243847: Deleting consensus metadata
I20260806 06:35:51.635494 319 catalog_manager.cc:4841] TS 8ea4545546aa4df2b96ddaeed3243847 (127.29.142.130:41743): tablet 3c2e3c8bab654e0489c1892b83e15d38 (table test-table [id=44e01cf041b942239f4feb27b0b73af0]) successfully deleted
I20260806 06:35:51.636077 1299 ts_tablet_manager.cc:1918] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:35:51.636165 1299 log.cc:1196] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/3c2e3c8bab654e0489c1892b83e15d38
I20260806 06:35:51.636451 1299 ts_tablet_manager.cc:1939] T 3c2e3c8bab654e0489c1892b83e15d38 P b1854ee34f734988be91601bb216ca40: Deleting consensus metadata
I20260806 06:35:51.636880 323 catalog_manager.cc:4841] TS b1854ee34f734988be91601bb216ca40 (127.29.142.131:43683): tablet 3c2e3c8bab654e0489c1892b83e15d38 (table test-table [id=44e01cf041b942239f4feb27b0b73af0]) successfully deleted
W20260806 06:35:51.646288 323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:44773 (error 108)
I20260806 06:35:51.646701 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 653
I20260806 06:35:51.655597 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:51.646878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:44773 (local address 127.0.0.1:54506)
0806 06:35:51.655432 (+ 8554us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:51.655544 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8501,"thread_start_us":72,"threads_started":1}
W20260806 06:35:51.655845 323 catalog_manager.cc:4584] TS 8f167c29d7bc4c1faaf2696c72e35658 (127.29.142.129:44773): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c2e3c8bab654e0489c1892b83e15d38: Network error: Client connection negotiation failed: client connection to 127.29.142.129:44773: connect: Connection refused (error 111)
I20260806 06:35:51.661604 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 853
I20260806 06:35:51.678786 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 32767
2026-08-06T06:35:51Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6391 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:51 dist-test-slave-3kk6 krb5kdc[1304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:51 dist-test-slave-3kk6 krb5kdc[1304](info): set up 2 sockets
Aug 06 06:35:51 dist-test-slave-3kk6 krb5kdc[1304](info): commencing operation
krb5kdc: starting...
W20260806 06:35:52.940213 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:35:53 dist-test-slave-3kk6 krb5kdc[1304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:35:53.085598 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:39535
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:39535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:53.169880 1326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:53.170116 1326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:53.170140 1326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:53.171391 1326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:53.171440 1326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:53.171455 1326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:53.171469 1326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:53.171481 1326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:53.173081 1326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:39535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:39535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:53.173363 1326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:53.173622 1326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:53.180104 1331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:53.180186 1334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:53.180167 1332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:53.180430 1326 server_base.cc:1034] running on GCE node
I20260806 06:35:53.180719 1326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:53.181061 1326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:53.182215 1326 hybrid_clock.cc:648] HybridClock initialized: now 1785998153182182 us; error 51 us; skew 500 ppm
Aug 06 06:35:53 dist-test-slave-3kk6 krb5kdc[1304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998153, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:53.190160 1326 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:35:53.190876 1326 webserver.cc:460] Webserver started at http://127.29.142.190:40219/ using document root <none> and password file <none>
I20260806 06:35:53.191133 1326 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:53.191187 1326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:53.191324 1326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:53.192514 1326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "56bba8d15acc47b6b8e354f5cd46bd93"
format_stamp: "Formatted at 2026-08-06 06:35:53 on dist-test-slave-3kk6"
server_key: "b7d39456a60b3bba359d8442aa949e30"
server_key_iv: "747acd3b3a7ec0f697b117ca6accc1c0"
server_key_version: "encryptionkey@0"
I20260806 06:35:53.192919 1326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "56bba8d15acc47b6b8e354f5cd46bd93"
format_stamp: "Formatted at 2026-08-06 06:35:53 on dist-test-slave-3kk6"
server_key: "b7d39456a60b3bba359d8442aa949e30"
server_key_iv: "747acd3b3a7ec0f697b117ca6accc1c0"
server_key_version: "encryptionkey@0"
I20260806 06:35:53.194393 1326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:53.195300 1347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:53.195500 1326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:35:53.195622 1326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "56bba8d15acc47b6b8e354f5cd46bd93"
format_stamp: "Formatted at 2026-08-06 06:35:53 on dist-test-slave-3kk6"
server_key: "b7d39456a60b3bba359d8442aa949e30"
server_key_iv: "747acd3b3a7ec0f697b117ca6accc1c0"
server_key_version: "encryptionkey@0"
I20260806 06:35:53.195708 1326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:53.209408 1326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:53.211573 1326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:53.211800 1326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:53.216249 1326 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:39535
I20260806 06:35:53.216312 1420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:39535 every 8 connection(s)
I20260806 06:35:53.216722 1326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:53.217669 1421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:53.220180 1421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93: Bootstrap starting.
I20260806 06:35:53.220765 1421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:53.221032 1421 log.cc:824] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:53.221652 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 1326
I20260806 06:35:53.221741 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:35:53.221853 1421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93: No bootstrap required, opened a new log
I20260806 06:35:53.221936 30266 external_mini_cluster.cc:1428] Setting key 9df9be7c8c2111901fb7ae6880beb41a
I20260806 06:35:53.224407 1421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56bba8d15acc47b6b8e354f5cd46bd93" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39535 } }
I20260806 06:35:53.224555 1421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:53.224579 1421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bba8d15acc47b6b8e354f5cd46bd93, State: Initialized, Role: FOLLOWER
I20260806 06:35:53.224717 1421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56bba8d15acc47b6b8e354f5cd46bd93" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39535 } }
I20260806 06:35:53.224799 1421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:53.224823 1421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:53.224905 1421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:53.225486 1421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56bba8d15acc47b6b8e354f5cd46bd93" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39535 } }
I20260806 06:35:53.225579 1421 leader_election.cc:302] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56bba8d15acc47b6b8e354f5cd46bd93; no voters:
I20260806 06:35:53.225788 1421 leader_election.cc:288] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:53.225817 1427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:53.225935 1427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [term 1 LEADER]: Becoming Leader. State: Replica: 56bba8d15acc47b6b8e354f5cd46bd93, State: Running, Role: LEADER
I20260806 06:35:53.226022 1427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56bba8d15acc47b6b8e354f5cd46bd93" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39535 } }
I20260806 06:35:53.226083 1421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:53.226342 1427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56bba8d15acc47b6b8e354f5cd46bd93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56bba8d15acc47b6b8e354f5cd46bd93" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39535 } } }
I20260806 06:35:53.226414 1427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:53.226558 1427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56bba8d15acc47b6b8e354f5cd46bd93. Latest consensus state: current_term: 1 leader_uuid: "56bba8d15acc47b6b8e354f5cd46bd93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56bba8d15acc47b6b8e354f5cd46bd93" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39535 } } }
I20260806 06:35:53.226620 1427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:53.227595 1432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:53.228168 1432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:35:53 dist-test-slave-3kk6 krb5kdc[1304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:53.230700 1432 catalog_manager.cc:1347] Generated new cluster ID: 15f24813ecf04b679862be39a6e70366
I20260806 06:35:53.230758 1432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:53.235023 1426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:53.222556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54424 (local address 127.29.142.190:39535)
0806 06:35:53.222760 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:53.222768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:53.222811 (+ 43us) server_negotiation.cc:407] Connection header received
0806 06:35:53.223330 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:53.223337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:53.223388 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:53.223508 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:53.224353 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.225847 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.227065 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.227830 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.227947 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:53.231086 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:53.231090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:53.231090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:53.231104 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:53.233221 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:53.233652 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:53.233656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:53.233656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:53.233779 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:53.234009 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:53.234010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:53.234011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:53.234345 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:53.234527 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:53.234617 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:35:53.234798 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":120,"threads_started":1}
I20260806 06:35:53.236994 1432 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:53.237718 1432 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:53.244539 1432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56bba8d15acc47b6b8e354f5cd46bd93: Generated new TSK 0
I20260806 06:35:53.244752 1432 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:35:53.275645 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39535
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:53.360787 1461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:53.361104 1461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:53.361151 1461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:53.362623 1461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:53.362706 1461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:53.362754 1461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:35:53.364233 1461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:53.364522 1461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:53.364778 1461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:35:53.370191 1461 server_base.cc:1034] running on GCE node
W20260806 06:35:53.370173 1469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:53.370173 1473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:53.370182 1471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:53.370625 1461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:53.370906 1461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:53.372085 1461 hybrid_clock.cc:648] HybridClock initialized: now 1785998153372070 us; error 35 us; skew 500 ppm
Aug 06 06:35:53 dist-test-slave-3kk6 krb5kdc[1304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998153, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:53.374766 1461 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:35:53.375283 1461 webserver.cc:460] Webserver started at http://127.29.142.129:35797/ using document root <none> and password file <none>
I20260806 06:35:53.375480 1461 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:53.375523 1461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:53.375612 1461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:53.376664 1461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "113db6e1613f489dac0f39b78f7c66f3"
format_stamp: "Formatted at 2026-08-06 06:35:53 on dist-test-slave-3kk6"
server_key: "c37399044b74a12729b1ce56fefb92fe"
server_key_iv: "af6e4e9cd7c0b7727048519b43177d6e"
server_key_version: "encryptionkey@0"
I20260806 06:35:53.376995 1461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "113db6e1613f489dac0f39b78f7c66f3"
format_stamp: "Formatted at 2026-08-06 06:35:53 on dist-test-slave-3kk6"
server_key: "c37399044b74a12729b1ce56fefb92fe"
server_key_iv: "af6e4e9cd7c0b7727048519b43177d6e"
server_key_version: "encryptionkey@0"
I20260806 06:35:53.378306 1461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:35:53.379118 1481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:53.379318 1461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:35:53.379433 1461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "113db6e1613f489dac0f39b78f7c66f3"
format_stamp: "Formatted at 2026-08-06 06:35:53 on dist-test-slave-3kk6"
server_key: "c37399044b74a12729b1ce56fefb92fe"
server_key_iv: "af6e4e9cd7c0b7727048519b43177d6e"
server_key_version: "encryptionkey@0"
I20260806 06:35:53.379515 1461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:53.398381 1461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:53.401178 1461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:53.401378 1461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:53.401674 1461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:53.402037 1461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:53.402097 1461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:53.402134 1461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:53.402168 1461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:53.408425 1461 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:36969
I20260806 06:35:53.408478 1639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:36969 every 8 connection(s)
I20260806 06:35:53.408880 1461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:35:53.411443 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 1461
I20260806 06:35:53.411553 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:35:53.411777 30266 external_mini_cluster.cc:1428] Setting key e959b32e615e8b0d039be47cd4d1b8d4
Aug 06 06:35:53 dist-test-slave-3kk6 krb5kdc[1304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998153, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:53.420614 1426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:53.409748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:51259 (local address 127.29.142.190:39535)
0806 06:35:53.409823 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:53.409825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:53.410017 (+ 192us) server_negotiation.cc:407] Connection header received
0806 06:35:53.410503 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:53.410504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:53.410514 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:53.410581 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:53.411983 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.412705 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.414244 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.414575 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.414689 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:53.417372 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:53.417374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:53.417374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:53.417381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:53.418972 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:53.419514 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:53.419515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:53.419516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:53.419577 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:53.419915 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:53.419916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:53.419916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:53.420030 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:53.420088 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:53.420454 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 06:35:53.420542 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:35:53.420995 1643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:53.409683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39535 (local address 127.29.142.129:51259)
0806 06:35:53.409873 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:53.409918 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:53.410414 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:53.410681 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:53.410685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:53.411452 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:53.411787 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:53.411794 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.412801 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.412802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:53.414151 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:53.414154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.414711 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.414712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:53.414828 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:53.415441 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:53.415451 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:53.417257 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:53.419062 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:53.419072 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:53.419074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:53.419403 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:53.419662 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:53.419663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:53.419663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:53.419835 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:53.420105 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:53.420111 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:53.420290 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 06:35:53.420424 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 06:35:53.420611 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
I20260806 06:35:53.421599 1640 heartbeater.cc:347] Connected to a master server at 127.29.142.190:39535
I20260806 06:35:53.421736 1640 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:53.421983 1640 heartbeater.cc:511] Master 127.29.142.190:39535 requested a full tablet report, sending...
I20260806 06:35:53.422588 1369 ts_manager.cc:194] Registered new tserver with Master: 113db6e1613f489dac0f39b78f7c66f3 (127.29.142.129:36969)
I20260806 06:35:53.423287 1369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:51259
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:35:53.441499 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39535
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:53.527908 1646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:53.528148 1646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:53.528178 1646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:53.529345 1646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:53.529385 1646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:53.529412 1646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:35:53.530889 1646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:53.531104 1646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:53.531313 1646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:53.536775 1663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:53.536814 1659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:53.536697 1657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:53.537113 1646 server_base.cc:1034] running on GCE node
I20260806 06:35:53.537411 1646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:53.537683 1646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:53.538858 1646 hybrid_clock.cc:648] HybridClock initialized: now 1785998153538849 us; error 54 us; skew 500 ppm
Aug 06 06:35:53 dist-test-slave-3kk6 krb5kdc[1304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998153, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:53.541682 1646 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:35:53.542126 1646 webserver.cc:460] Webserver started at http://127.29.142.130:41877/ using document root <none> and password file <none>
I20260806 06:35:53.542397 1646 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:53.542447 1646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:53.542618 1646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:53.543784 1646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "308f895a7d7d4348bb67be958c5eee5a"
format_stamp: "Formatted at 2026-08-06 06:35:53 on dist-test-slave-3kk6"
server_key: "f5f9a68f9f65d6edd48905f8c275931c"
server_key_iv: "8cb8c5ccedbc5a1b0785b1303f93bf79"
server_key_version: "encryptionkey@0"
I20260806 06:35:53.544100 1646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "308f895a7d7d4348bb67be958c5eee5a"
format_stamp: "Formatted at 2026-08-06 06:35:53 on dist-test-slave-3kk6"
server_key: "f5f9a68f9f65d6edd48905f8c275931c"
server_key_iv: "8cb8c5ccedbc5a1b0785b1303f93bf79"
server_key_version: "encryptionkey@0"
I20260806 06:35:53.545450 1646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:35:53.546212 1672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:53.546411 1646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:53.546521 1646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "308f895a7d7d4348bb67be958c5eee5a"
format_stamp: "Formatted at 2026-08-06 06:35:53 on dist-test-slave-3kk6"
server_key: "f5f9a68f9f65d6edd48905f8c275931c"
server_key_iv: "8cb8c5ccedbc5a1b0785b1303f93bf79"
server_key_version: "encryptionkey@0"
I20260806 06:35:53.546617 1646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:53.555431 1646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:53.558049 1646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:53.558238 1646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:53.558495 1646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:53.558907 1646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:53.558974 1646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:53.559020 1646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:53.559048 1646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:53.565387 1646 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:43353
I20260806 06:35:53.565474 1828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:43353 every 8 connection(s)
I20260806 06:35:53.565841 1646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:35:53.567415 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 1646
I20260806 06:35:53.567536 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:35:53.567775 30266 external_mini_cluster.cc:1428] Setting key dfd38ca5b54ffcc7fea32fd2e85fb936
Aug 06 06:35:53 dist-test-slave-3kk6 krb5kdc[1304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998153, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:53.577553 1426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:53.566752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:52795 (local address 127.29.142.190:39535)
0806 06:35:53.566823 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:53.566826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:53.567012 (+ 186us) server_negotiation.cc:407] Connection header received
0806 06:35:53.567581 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:53.567583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:53.567599 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:53.567679 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:53.568957 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.569711 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.571375 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.571565 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.571674 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:53.574228 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:53.574229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:53.574230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:53.574239 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:53.575915 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:53.576475 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:53.576476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:53.576476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:53.576540 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:53.576865 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:53.576865 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:53.576865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:53.576974 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:53.577026 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:53.577415 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 06:35:53.577481 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
I20260806 06:35:53.577854 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:53.566639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39535 (local address 127.29.142.130:52795)
0806 06:35:53.566875 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:53.566916 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:53.567427 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:53.567741 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:53.567744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:53.568624 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:53.568845 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:53.568851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.569844 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.569845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:53.571264 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:53.571269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.571688 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.571689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:53.571827 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:53.572445 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:53.572453 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:53.574111 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:53.576021 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:53.576029 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:53.576032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:53.576379 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:53.576614 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:53.576615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:53.576616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:53.576786 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:53.577046 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:53.577054 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:53.577248 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 06:35:53.577390 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 06:35:53.577581 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260806 06:35:53.578514 1829 heartbeater.cc:347] Connected to a master server at 127.29.142.190:39535
I20260806 06:35:53.578622 1829 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:53.578871 1829 heartbeater.cc:511] Master 127.29.142.190:39535 requested a full tablet report, sending...
I20260806 06:35:53.579303 1369 ts_manager.cc:194] Registered new tserver with Master: 308f895a7d7d4348bb67be958c5eee5a (127.29.142.130:43353)
I20260806 06:35:53.579651 1369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:52795
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:35:53.597599 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39535
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:53.684078 1837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:53.684296 1837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:53.684322 1837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:53.685415 1837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:53.685451 1837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:53.685478 1837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:35:53.686976 1837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:53.687180 1837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:53.687373 1837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:53.692713 1850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:53.692730 1845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:53.692852 1844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:53.692889 1837 server_base.cc:1034] running on GCE node
I20260806 06:35:53.693244 1837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:53.693507 1837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:53.694708 1837 hybrid_clock.cc:648] HybridClock initialized: now 1785998153694689 us; error 54 us; skew 500 ppm
Aug 06 06:35:53 dist-test-slave-3kk6 krb5kdc[1304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998153, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:53.697443 1837 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:35:53.697937 1837 webserver.cc:460] Webserver started at http://127.29.142.131:42541/ using document root <none> and password file <none>
I20260806 06:35:53.698174 1837 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:53.698263 1837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:53.698403 1837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:53.699664 1837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "ce8046680e194dda9fccb2c4f65ca945"
format_stamp: "Formatted at 2026-08-06 06:35:53 on dist-test-slave-3kk6"
server_key: "26319c0ff9b25a64494996ba6f0715e1"
server_key_iv: "037d188c4229de22f5e884d6b639c8da"
server_key_version: "encryptionkey@0"
I20260806 06:35:53.700054 1837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "ce8046680e194dda9fccb2c4f65ca945"
format_stamp: "Formatted at 2026-08-06 06:35:53 on dist-test-slave-3kk6"
server_key: "26319c0ff9b25a64494996ba6f0715e1"
server_key_iv: "037d188c4229de22f5e884d6b639c8da"
server_key_version: "encryptionkey@0"
I20260806 06:35:53.701499 1837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:35:53.702291 1859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:53.702512 1837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:53.702620 1837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "ce8046680e194dda9fccb2c4f65ca945"
format_stamp: "Formatted at 2026-08-06 06:35:53 on dist-test-slave-3kk6"
server_key: "26319c0ff9b25a64494996ba6f0715e1"
server_key_iv: "037d188c4229de22f5e884d6b639c8da"
server_key_version: "encryptionkey@0"
I20260806 06:35:53.702797 1837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:53.712530 1837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:53.715076 1837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:53.715263 1837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:53.715543 1837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:53.715896 1837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:53.715952 1837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:53.715988 1837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:53.716030 1837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:53.722347 1837 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:33679
I20260806 06:35:53.722398 2020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:33679 every 8 connection(s)
I20260806 06:35:53.722846 1837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:35:53.723696 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 1837
I20260806 06:35:53.723836 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:35:53.724040 30266 external_mini_cluster.cc:1428] Setting key 0c1bb625d398704e6363bc90452d3fcb
Aug 06 06:35:53 dist-test-slave-3kk6 krb5kdc[1304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998153, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:53.733589 1426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:53.723820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:37671 (local address 127.29.142.190:39535)
0806 06:35:53.723883 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:53.723885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:53.724117 (+ 232us) server_negotiation.cc:407] Connection header received
0806 06:35:53.724804 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:53.724805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:53.724818 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:53.724856 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:53.725939 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.726464 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.727733 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.727971 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.728055 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:53.730490 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:53.730492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:53.730492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:53.730503 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:53.731961 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:53.732419 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:53.732420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:53.732421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:53.732491 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:53.732888 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:53.732889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:53.732889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:53.733031 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:53.733104 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:53.733419 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 06:35:53.733515 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:35:53.733747 2024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:53.723769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39535 (local address 127.29.142.131:37671)
0806 06:35:53.723961 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:53.724007 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:53.724683 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:53.724941 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:53.724946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:53.725558 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:53.725839 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:53.725845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.726578 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.726580 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:53.727620 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:53.727623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:53.728079 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:53.728080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:53.728210 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:53.728873 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:53.728883 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:53.730371 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:53.732063 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:53.732073 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:53.732076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:53.732329 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:53.732566 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:53.732567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:53.732567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:53.732771 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:53.733112 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:53.733118 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:53.733253 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 06:35:53.733351 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 06:35:53.733490 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260806 06:35:53.734331 2021 heartbeater.cc:347] Connected to a master server at 127.29.142.190:39535
I20260806 06:35:53.734459 2021 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:53.734767 2021 heartbeater.cc:511] Master 127.29.142.190:39535 requested a full tablet report, sending...
I20260806 06:35:53.735242 1369 ts_manager.cc:194] Registered new tserver with Master: ce8046680e194dda9fccb2c4f65ca945 (127.29.142.131:33679)
I20260806 06:35:53.735790 1369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:37671
I20260806 06:35:53.736289 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:35:53.750008 1426 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:35:53.748981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54430 (local address 127.29.142.190:39535)
0806 06:35:53.749049 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:53.749051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:53.749137 (+ 86us) server_negotiation.cc:407] Connection header received
0806 06:35:53.749196 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:53.749197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:53.749206 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:53.749276 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:53.749946 (+ 670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54430: BlockingRecv error: recv got EOF from 127.0.0.1:54430 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:35:53.751041 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 1461
I20260806 06:35:53.786777 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 1646
I20260806 06:35:53.810132 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 1837
I20260806 06:35:53.828374 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 1326
2026-08-06T06:35:53Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2158 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:53 dist-test-slave-3kk6 krb5kdc[2038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:53 dist-test-slave-3kk6 krb5kdc[2038](info): set up 2 sockets
Aug 06 06:35:53 dist-test-slave-3kk6 krb5kdc[2038](info): commencing operation
krb5kdc: starting...
W20260806 06:35:55.104702 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:35:55 dist-test-slave-3kk6 krb5kdc[2038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:35:55.233230 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:38497
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:37531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:38497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:55.318938 2054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:55.319178 2054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:55.319202 2054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:55.320346 2054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:55.320396 2054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:55.320411 2054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:55.320425 2054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:55.320436 2054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:55.321918 2054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:37531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:38497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:38497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:55.322132 2054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:55.322317 2054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:55.327490 2061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:55.327457 2063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:55.327457 2060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:55.327766 2054 server_base.cc:1034] running on GCE node
I20260806 06:35:55.328048 2054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:55.328374 2054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:55.329555 2054 hybrid_clock.cc:648] HybridClock initialized: now 1785998155329547 us; error 37 us; skew 500 ppm
Aug 06 06:35:55 dist-test-slave-3kk6 krb5kdc[2038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998155, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:55.332263 2054 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:35:55.332706 2054 webserver.cc:460] Webserver started at http://127.29.142.190:41139/ using document root <none> and password file <none>
I20260806 06:35:55.332906 2054 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:55.332947 2054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:55.333041 2054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:55.334117 2054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "e4ce31975fc642ebb59c37dfb30764d9"
format_stamp: "Formatted at 2026-08-06 06:35:55 on dist-test-slave-3kk6"
server_key: "2d418a7389a3de53e17b2dde9e446223"
server_key_iv: "8cdfc1ce594413d8be7ae380dd449d31"
server_key_version: "encryptionkey@0"
I20260806 06:35:55.334456 2054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "e4ce31975fc642ebb59c37dfb30764d9"
format_stamp: "Formatted at 2026-08-06 06:35:55 on dist-test-slave-3kk6"
server_key: "2d418a7389a3de53e17b2dde9e446223"
server_key_iv: "8cdfc1ce594413d8be7ae380dd449d31"
server_key_version: "encryptionkey@0"
I20260806 06:35:55.335929 2054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:35:55.336747 2074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:55.336931 2054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:55.337025 2054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "e4ce31975fc642ebb59c37dfb30764d9"
format_stamp: "Formatted at 2026-08-06 06:35:55 on dist-test-slave-3kk6"
server_key: "2d418a7389a3de53e17b2dde9e446223"
server_key_iv: "8cdfc1ce594413d8be7ae380dd449d31"
server_key_version: "encryptionkey@0"
I20260806 06:35:55.337086 2054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:55.373992 2054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:55.376247 2054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:55.376379 2054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:55.380671 2054 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:38497
I20260806 06:35:55.380743 2147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:38497 every 8 connection(s)
I20260806 06:35:55.381315 2054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:55.382078 2148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:55.384588 2148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9: Bootstrap starting.
I20260806 06:35:55.385229 2148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:55.385545 2148 log.cc:824] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:55.386324 2148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9: No bootstrap required, opened a new log
I20260806 06:35:55.388032 2148 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ce31975fc642ebb59c37dfb30764d9" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38497 } }
I20260806 06:35:55.388211 2148 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:55.388278 2148 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4ce31975fc642ebb59c37dfb30764d9, State: Initialized, Role: FOLLOWER
I20260806 06:35:55.388455 2148 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ce31975fc642ebb59c37dfb30764d9" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38497 } }
I20260806 06:35:55.388545 2148 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:55.388590 2148 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:55.388643 2148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:55.388994 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 2054
I20260806 06:35:55.389089 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:35:55.389299 30266 external_mini_cluster.cc:1428] Setting key 076ba059a389f479cb5107f4b46e4809
I20260806 06:35:55.389886 2148 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ce31975fc642ebb59c37dfb30764d9" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38497 } }
I20260806 06:35:55.390019 2148 leader_election.cc:302] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4ce31975fc642ebb59c37dfb30764d9; no voters:
I20260806 06:35:55.390244 2148 leader_election.cc:288] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:55.390275 2154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:55.390398 2154 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [term 1 LEADER]: Becoming Leader. State: Replica: e4ce31975fc642ebb59c37dfb30764d9, State: Running, Role: LEADER
I20260806 06:35:55.390537 2148 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:55.390516 2154 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ce31975fc642ebb59c37dfb30764d9" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38497 } }
I20260806 06:35:55.390908 2154 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4ce31975fc642ebb59c37dfb30764d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ce31975fc642ebb59c37dfb30764d9" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38497 } } }
I20260806 06:35:55.390969 2154 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:55.391073 2154 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4ce31975fc642ebb59c37dfb30764d9. Latest consensus state: current_term: 1 leader_uuid: "e4ce31975fc642ebb59c37dfb30764d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4ce31975fc642ebb59c37dfb30764d9" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38497 } } }
I20260806 06:35:55.391108 2154 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:55.391786 2159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:55.392426 2159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:35:55 dist-test-slave-3kk6 krb5kdc[2038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:55.399153 2159 catalog_manager.cc:1347] Generated new cluster ID: 927e3205e1824733b9078abe907bacd8
I20260806 06:35:55.399230 2159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:55.400898 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:55.389886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41224 (local address 127.29.142.190:38497)
0806 06:35:55.390148 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:55.390159 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:55.390200 (+ 41us) server_negotiation.cc:407] Connection header received
0806 06:35:55.390341 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:55.390350 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:55.390415 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:55.390608 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:55.391494 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:55.393046 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:55.393836 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:55.394261 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:55.394407 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:55.397292 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:55.397296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:55.397296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:55.397307 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:55.399357 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:55.399737 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:55.399740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:55.399740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:55.399850 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:55.400076 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:55.400077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:55.400077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:55.400322 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:55.400506 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:55.400577 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 06:35:55.400725 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":123,"threads_started":1}
I20260806 06:35:55.401840 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:35:55.425149 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:55.420489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41232 (local address 127.29.142.190:38497)
0806 06:35:55.420562 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:55.420565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:55.420708 (+ 143us) server_negotiation.cc:407] Connection header received
0806 06:35:55.420770 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:55.420771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:55.420783 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:55.420866 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:55.421353 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:55.422145 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:55.423288 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:55.423624 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:55.423704 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:55.424549 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:55.424551 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:55.424552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:35:55.424825 (+ 273us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:55.424846 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:35:55.424854 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:55.424928 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:55.425031 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 06:35:55.425057 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:35:55.426275 2159 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:55.427076 2159 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:55.435101 2159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4ce31975fc642ebb59c37dfb30764d9: Generated new TSK 0
I20260806 06:35:55.435328 2159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:35:55.442530 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 2054
2026-08-06T06:35:55Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1602 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:55 dist-test-slave-3kk6 krb5kdc[2194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:55 dist-test-slave-3kk6 krb5kdc[2194](info): set up 2 sockets
Aug 06 06:35:55 dist-test-slave-3kk6 krb5kdc[2194](info): commencing operation
krb5kdc: starting...
W20260806 06:35:56.683128 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:35:56 dist-test-slave-3kk6 krb5kdc[2194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:35:56.818589 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:36751
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:36751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:56.904714 2216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:56.905009 2216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:56.905051 2216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:56.906256 2216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:56.906335 2216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:56.906359 2216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:56.906383 2216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:56.906409 2216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:56.908094 2216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:36751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:36751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:56.908409 2216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:56.908660 2216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:56.913936 2225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:56.913936 2227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:56.914024 2231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:56.914330 2216 server_base.cc:1034] running on GCE node
I20260806 06:35:56.914721 2216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:56.915174 2216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:56.916369 2216 hybrid_clock.cc:648] HybridClock initialized: now 1785998156916355 us; error 52 us; skew 500 ppm
Aug 06 06:35:56 dist-test-slave-3kk6 krb5kdc[2194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998156, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:56.919325 2216 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:35:56.919802 2216 webserver.cc:460] Webserver started at http://127.29.142.190:46547/ using document root <none> and password file <none>
I20260806 06:35:56.920063 2216 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:56.920135 2216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:56.920290 2216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:56.921432 2216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "e07ad9e3432c4ceab5bda5c474ee73b2"
format_stamp: "Formatted at 2026-08-06 06:35:56 on dist-test-slave-3kk6"
server_key: "154ca25a52b3c071625ea9859f7b24c5"
server_key_iv: "1650577552f20754b0a69cf885f41cc8"
server_key_version: "encryptionkey@0"
I20260806 06:35:56.921831 2216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "e07ad9e3432c4ceab5bda5c474ee73b2"
format_stamp: "Formatted at 2026-08-06 06:35:56 on dist-test-slave-3kk6"
server_key: "154ca25a52b3c071625ea9859f7b24c5"
server_key_iv: "1650577552f20754b0a69cf885f41cc8"
server_key_version: "encryptionkey@0"
I20260806 06:35:56.923388 2216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:35:56.924178 2240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:56.924378 2216 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:56.924487 2216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "e07ad9e3432c4ceab5bda5c474ee73b2"
format_stamp: "Formatted at 2026-08-06 06:35:56 on dist-test-slave-3kk6"
server_key: "154ca25a52b3c071625ea9859f7b24c5"
server_key_iv: "1650577552f20754b0a69cf885f41cc8"
server_key_version: "encryptionkey@0"
I20260806 06:35:56.924579 2216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:56.936547 2216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:56.939435 2216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:56.939652 2216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:56.943966 2216 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:36751
I20260806 06:35:56.944011 2321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:36751 every 8 connection(s)
I20260806 06:35:56.944424 2216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:56.944784 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 2216
I20260806 06:35:56.944919 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:35:56.945192 30266 external_mini_cluster.cc:1428] Setting key 3f6688707899ea5b487483afb5510eef
I20260806 06:35:56.946184 2322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:56.949672 2322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2: Bootstrap starting.
I20260806 06:35:56.950435 2322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:56.950892 2322 log.cc:824] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:56.951926 2322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2: No bootstrap required, opened a new log
Aug 06 06:35:56 dist-test-slave-3kk6 krb5kdc[2194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:56.954751 2322 raft_consensus.cc:348] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ad9e3432c4ceab5bda5c474ee73b2" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36751 } }
I20260806 06:35:56.954989 2322 raft_consensus.cc:374] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:56.955051 2322 raft_consensus.cc:729] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07ad9e3432c4ceab5bda5c474ee73b2, State: Initialized, Role: FOLLOWER
I20260806 06:35:56.955225 2322 consensus_queue.cc:260] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ad9e3432c4ceab5bda5c474ee73b2" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36751 } }
I20260806 06:35:56.955348 2322 raft_consensus.cc:388] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:56.955387 2322 raft_consensus.cc:482] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:56.955440 2322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:56.956336 2322 raft_consensus.cc:504] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ad9e3432c4ceab5bda5c474ee73b2" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36751 } }
I20260806 06:35:56.956489 2322 leader_election.cc:302] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e07ad9e3432c4ceab5bda5c474ee73b2; no voters:
I20260806 06:35:56.956746 2322 leader_election.cc:288] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:56.956796 2329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:56.956979 2329 raft_consensus.cc:686] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [term 1 LEADER]: Becoming Leader. State: Replica: e07ad9e3432c4ceab5bda5c474ee73b2, State: Running, Role: LEADER
I20260806 06:35:56.957080 2322 sys_catalog.cc:564] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:56.957134 2329 consensus_queue.cc:237] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ad9e3432c4ceab5bda5c474ee73b2" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36751 } }
I20260806 06:35:56.957528 2330 sys_catalog.cc:455] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e07ad9e3432c4ceab5bda5c474ee73b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ad9e3432c4ceab5bda5c474ee73b2" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36751 } } }
I20260806 06:35:56.957618 2330 sys_catalog.cc:458] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:56.957779 2330 sys_catalog.cc:455] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e07ad9e3432c4ceab5bda5c474ee73b2. Latest consensus state: current_term: 1 leader_uuid: "e07ad9e3432c4ceab5bda5c474ee73b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ad9e3432c4ceab5bda5c474ee73b2" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36751 } } }
I20260806 06:35:56.957877 2330 sys_catalog.cc:458] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:56.958571 2339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:56.958714 2327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:56.945898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56676 (local address 127.29.142.190:36751)
0806 06:35:56.946082 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:56.946091 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:56.946118 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:35:56.946630 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:56.946638 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:56.946755 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:56.946896 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:56.947812 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:56.948732 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:56.949942 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:56.950496 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:56.950727 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:56.953726 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:56.953733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:56.953733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:56.953763 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:56.956205 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:56.956722 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:56.956727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:56.956728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:56.956902 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:56.957162 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:56.957163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:56.957163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:56.957520 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:56.957661 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:56.958223 (+ 562us) server_negotiation.cc:299] Negotiation successful
0806 06:35:56.958404 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260806 06:35:56.959386 2339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:56.960973 2339 catalog_manager.cc:1347] Generated new cluster ID: 235f37c71d684f5a82ce52f82273cf29
I20260806 06:35:56.961030 2339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:35:56.971958 2339 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:56.972519 2339 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:56.978641 2339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e07ad9e3432c4ceab5bda5c474ee73b2: Generated new TSK 0
I20260806 06:35:56.978929 2339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:35:57.014453 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:36751
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:57.119081 2360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:57.119311 2360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:57.119336 2360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:57.120503 2360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:57.120551 2360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:57.120579 2360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:35:57.122105 2360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:36751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:57.122364 2360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:57.122609 2360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:57.127832 2365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:57.127941 2366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:57.127993 2360 server_base.cc:1034] running on GCE node
W20260806 06:35:57.127832 2369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:57.128307 2360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:57.128576 2360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:57.129733 2360 hybrid_clock.cc:648] HybridClock initialized: now 1785998157129713 us; error 42 us; skew 500 ppm
Aug 06 06:35:57 dist-test-slave-3kk6 krb5kdc[2194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998157, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:57.132622 2360 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:35:57.133209 2360 webserver.cc:460] Webserver started at http://127.29.142.129:33133/ using document root <none> and password file <none>
I20260806 06:35:57.133481 2360 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:57.133554 2360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:57.133713 2360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:57.134914 2360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "dde018cb6f634f2a9956d82506d77279"
format_stamp: "Formatted at 2026-08-06 06:35:57 on dist-test-slave-3kk6"
server_key: "4c4bfe5cf3c0a16524e602096a03976c"
server_key_iv: "745e58b34e11d3ea0fb5f6d552405a9d"
server_key_version: "encryptionkey@0"
I20260806 06:35:57.135293 2360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "dde018cb6f634f2a9956d82506d77279"
format_stamp: "Formatted at 2026-08-06 06:35:57 on dist-test-slave-3kk6"
server_key: "4c4bfe5cf3c0a16524e602096a03976c"
server_key_iv: "745e58b34e11d3ea0fb5f6d552405a9d"
server_key_version: "encryptionkey@0"
I20260806 06:35:57.136700 2360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:57.137497 2377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:57.137681 2360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:57.137795 2360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "dde018cb6f634f2a9956d82506d77279"
format_stamp: "Formatted at 2026-08-06 06:35:57 on dist-test-slave-3kk6"
server_key: "4c4bfe5cf3c0a16524e602096a03976c"
server_key_iv: "745e58b34e11d3ea0fb5f6d552405a9d"
server_key_version: "encryptionkey@0"
I20260806 06:35:57.137887 2360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:57.153057 2360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:57.155509 2360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:57.155705 2360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:57.156002 2360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:57.156361 2360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:57.156418 2360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:57.156456 2360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:57.156502 2360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:57.163041 2360 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:41963
I20260806 06:35:57.163086 2532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:41963 every 8 connection(s)
I20260806 06:35:57.163456 2360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:35:57 dist-test-slave-3kk6 krb5kdc[2194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998157, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:57.171257 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 2360
I20260806 06:35:57.171382 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:35:57.171617 30266 external_mini_cluster.cc:1428] Setting key 6661d476d9ea8b4f0ecc28234029bd46
I20260806 06:35:57.173592 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:57.164251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:36751 (local address 127.29.142.129:36589)
0806 06:35:57.164446 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:57.164491 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:57.164998 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:57.165218 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:57.165221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:57.165817 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:57.166020 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:57.166025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.166790 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.166791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:57.167615 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:57.167618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.167968 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.167969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:57.168086 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:57.168586 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:57.168595 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:57.170155 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:57.172054 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:57.172061 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:57.172064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:57.172408 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:57.172620 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:57.172621 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:57.172621 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:57.172740 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:57.173044 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:57.173049 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:57.173188 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 06:35:57.173287 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 06:35:57.173418 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":114,"threads_started":1}
I20260806 06:35:57.173589 2327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:57.164282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:36589 (local address 127.29.142.190:36751)
0806 06:35:57.164346 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:57.164348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:57.164582 (+ 234us) server_negotiation.cc:407] Connection header received
0806 06:35:57.165083 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:57.165084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:57.165094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:57.165150 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:57.166104 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.166609 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.167697 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.167891 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.167945 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:57.170273 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:57.170275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:57.170276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:57.170284 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:57.171958 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:57.172481 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:57.172483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:57.172483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:57.172553 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:57.172809 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:57.172810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:57.172811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:57.172975 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:57.173049 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:57.173336 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 06:35:57.173443 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:35:57.174363 2533 heartbeater.cc:347] Connected to a master server at 127.29.142.190:36751
I20260806 06:35:57.174499 2533 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:57.174754 2533 heartbeater.cc:511] Master 127.29.142.190:36751 requested a full tablet report, sending...
I20260806 06:35:57.175331 2264 ts_manager.cc:194] Registered new tserver with Master: dde018cb6f634f2a9956d82506d77279 (127.29.142.129:41963)
I20260806 06:35:57.175987 2264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:36589
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:35:57.201506 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:36751
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:57.288358 2546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:57.288628 2546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:57.288671 2546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:57.289855 2546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:57.289933 2546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:57.289978 2546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:35:57.291541 2546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:36751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:57.291853 2546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:57.292110 2546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:57.297631 2556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:57.297631 2553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:57.297950 2546 server_base.cc:1034] running on GCE node
W20260806 06:35:57.297652 2552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:57.298202 2546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:57.298480 2546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:57.299649 2546 hybrid_clock.cc:648] HybridClock initialized: now 1785998157299643 us; error 53 us; skew 500 ppm
Aug 06 06:35:57 dist-test-slave-3kk6 krb5kdc[2194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998157, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:57.302515 2546 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:35:57.303176 2546 webserver.cc:460] Webserver started at http://127.29.142.130:34581/ using document root <none> and password file <none>
I20260806 06:35:57.303438 2546 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:57.303517 2546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:57.303683 2546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:57.304853 2546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "c7406cbfcfe049fcb220b85c9346ff7f"
format_stamp: "Formatted at 2026-08-06 06:35:57 on dist-test-slave-3kk6"
server_key: "2e128a892ecc323c42485adc22ce8808"
server_key_iv: "f4b0ca93c3c92fe19999c659c7c510ff"
server_key_version: "encryptionkey@0"
I20260806 06:35:57.305259 2546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "c7406cbfcfe049fcb220b85c9346ff7f"
format_stamp: "Formatted at 2026-08-06 06:35:57 on dist-test-slave-3kk6"
server_key: "2e128a892ecc323c42485adc22ce8808"
server_key_iv: "f4b0ca93c3c92fe19999c659c7c510ff"
server_key_version: "encryptionkey@0"
I20260806 06:35:57.306825 2546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:57.307644 2565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:57.307859 2546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:57.307971 2546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "c7406cbfcfe049fcb220b85c9346ff7f"
format_stamp: "Formatted at 2026-08-06 06:35:57 on dist-test-slave-3kk6"
server_key: "2e128a892ecc323c42485adc22ce8808"
server_key_iv: "f4b0ca93c3c92fe19999c659c7c510ff"
server_key_version: "encryptionkey@0"
I20260806 06:35:57.308065 2546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:57.319905 2546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:57.322206 2546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:57.322391 2546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:57.322719 2546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:57.323084 2546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:57.323141 2546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:57.323186 2546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:57.323213 2546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:57.329777 2546 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:41425
I20260806 06:35:57.329823 2723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:41425 every 8 connection(s)
I20260806 06:35:57.330204 2546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:35:57 dist-test-slave-3kk6 krb5kdc[2194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998157, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:57.337726 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 2546
I20260806 06:35:57.337872 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:35:57.338091 30266 external_mini_cluster.cc:1428] Setting key 0438a0a304e61816686270f608e4a222
I20260806 06:35:57.340983 2327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:57.331063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:39975 (local address 127.29.142.190:36751)
0806 06:35:57.331134 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:57.331136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:57.331377 (+ 241us) server_negotiation.cc:407] Connection header received
0806 06:35:57.331962 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:57.331964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:57.331978 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:57.332030 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:57.333039 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.333564 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.334607 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.334871 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.334962 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:57.337253 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:57.337255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:57.337255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:57.337262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:57.339140 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:57.339705 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:57.339706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:57.339706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:57.339790 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:57.340091 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:57.340091 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:57.340092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:57.340199 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:57.340251 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:57.340644 (+ 393us) server_negotiation.cc:299] Negotiation successful
0806 06:35:57.340780 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:35:57.341073 2728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:57.331037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:36751 (local address 127.29.142.130:39975)
0806 06:35:57.331234 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:57.331285 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:57.331842 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:57.332108 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:57.332111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:57.332719 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:57.332932 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:57.332937 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.333672 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.333673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:57.334510 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:57.334512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.334975 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.334976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:57.335101 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:57.335682 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:57.335691 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:57.337159 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:57.339231 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:57.339240 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:57.339244 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:57.339619 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:57.339860 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:57.339861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:57.339861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:57.340023 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:57.340265 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:57.340272 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:57.340459 (+ 187us) client_negotiation.cc:769] Sending connection context
0806 06:35:57.340582 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 06:35:57.340815 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":101,"threads_started":1}
I20260806 06:35:57.341941 2724 heartbeater.cc:347] Connected to a master server at 127.29.142.190:36751
I20260806 06:35:57.342084 2724 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:57.342308 2724 heartbeater.cc:511] Master 127.29.142.190:36751 requested a full tablet report, sending...
I20260806 06:35:57.342767 2264 ts_manager.cc:194] Registered new tserver with Master: c7406cbfcfe049fcb220b85c9346ff7f (127.29.142.130:41425)
I20260806 06:35:57.343181 2264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:39975
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:35:57.367543 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:36751
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:57.457055 2732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:57.457350 2732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:57.457384 2732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:57.458912 2732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:57.458968 2732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:57.458998 2732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:35:57.460510 2732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:36751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:57.460737 2732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:57.460929 2732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:57.466591 2738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:57.466724 2732 server_base.cc:1034] running on GCE node
W20260806 06:35:57.466593 2741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:57.466876 2739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:57.467114 2732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:57.467429 2732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:57.468590 2732 hybrid_clock.cc:648] HybridClock initialized: now 1785998157468564 us; error 52 us; skew 500 ppm
Aug 06 06:35:57 dist-test-slave-3kk6 krb5kdc[2194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998157, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:57.471812 2732 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:35:57.472383 2732 webserver.cc:460] Webserver started at http://127.29.142.131:38111/ using document root <none> and password file <none>
I20260806 06:35:57.472666 2732 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:57.472718 2732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:57.472873 2732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:57.474085 2732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "a4de70bb3b3b4af3ae0e6ea7b79dfdbc"
format_stamp: "Formatted at 2026-08-06 06:35:57 on dist-test-slave-3kk6"
server_key: "698c60064eece792b6ec7f7c78cd7784"
server_key_iv: "197eb5ae0bf9c09f509726675ba22447"
server_key_version: "encryptionkey@0"
I20260806 06:35:57.474467 2732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "a4de70bb3b3b4af3ae0e6ea7b79dfdbc"
format_stamp: "Formatted at 2026-08-06 06:35:57 on dist-test-slave-3kk6"
server_key: "698c60064eece792b6ec7f7c78cd7784"
server_key_iv: "197eb5ae0bf9c09f509726675ba22447"
server_key_version: "encryptionkey@0"
I20260806 06:35:57.475962 2732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:35:57.476784 2751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:57.477006 2732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:57.477106 2732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "a4de70bb3b3b4af3ae0e6ea7b79dfdbc"
format_stamp: "Formatted at 2026-08-06 06:35:57 on dist-test-slave-3kk6"
server_key: "698c60064eece792b6ec7f7c78cd7784"
server_key_iv: "197eb5ae0bf9c09f509726675ba22447"
server_key_version: "encryptionkey@0"
I20260806 06:35:57.477185 2732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:57.498102 2732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:57.500669 2732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:57.500887 2732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:57.501188 2732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:57.501562 2732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:57.501621 2732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:57.501667 2732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:57.501695 2732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:57.507936 2732 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:35641
I20260806 06:35:57.507977 2895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:35641 every 8 connection(s)
I20260806 06:35:57.508345 2732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:35:57.513998 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 2732
I20260806 06:35:57.514120 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:35:57.514312 30266 external_mini_cluster.cc:1428] Setting key 43a64a2c64c6cdb89cc6555652e75dae
Aug 06 06:35:57 dist-test-slave-3kk6 krb5kdc[2194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998157, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:57.519196 2327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:57.509251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:46253 (local address 127.29.142.190:36751)
0806 06:35:57.509333 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:57.509336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:57.509502 (+ 166us) server_negotiation.cc:407] Connection header received
0806 06:35:57.510076 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:57.510077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:57.510091 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:57.510144 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:57.511294 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.511827 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.512922 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.513137 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.513225 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:57.516335 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:57.516336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:57.516337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:57.516344 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:57.517769 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:57.518201 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:57.518202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:57.518202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:57.518256 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:57.518564 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:57.518565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:57.518565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:57.518742 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:57.518797 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:57.519059 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 06:35:57.519125 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:35:57.519445 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:57.509149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:36751 (local address 127.29.142.131:46253)
0806 06:35:57.509368 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:57.509413 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:57.509957 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:57.510220 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:57.510224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:57.510926 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:57.511190 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:57.511196 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.511936 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.511938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:57.512815 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:57.512818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.513238 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.513239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:57.513372 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:57.513939 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:57.513951 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:57.516231 (+ 2280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:57.517855 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:57.517863 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:57.517865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:57.518129 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:57.518374 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:57.518375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:57.518376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:57.518499 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:57.518795 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:57.518801 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:57.518934 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 06:35:57.519029 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 06:35:57.519173 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260806 06:35:57.520004 2896 heartbeater.cc:347] Connected to a master server at 127.29.142.190:36751
I20260806 06:35:57.520126 2896 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:57.520359 2896 heartbeater.cc:511] Master 127.29.142.190:36751 requested a full tablet report, sending...
I20260806 06:35:57.520903 2264 ts_manager.cc:194] Registered new tserver with Master: a4de70bb3b3b4af3ae0e6ea7b79dfdbc (127.29.142.131:35641)
I20260806 06:35:57.521337 2264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:46253
I20260806 06:35:57.527120 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:35:57 dist-test-slave-3kk6 krb5kdc[2194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998157, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 06:35:57 dist-test-slave-3kk6 krb5kdc[2194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998157, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:57.553551 2327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:57.545185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.29.142.190:36751)
0806 06:35:57.545310 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:57.545312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:57.545358 (+ 46us) server_negotiation.cc:407] Connection header received
0806 06:35:57.545431 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:57.545432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:57.545444 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:57.545504 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:57.546389 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.547245 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.548006 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:57.548201 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:57.548272 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:57.550863 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:57.550865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:57.550866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:57.550873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:57.552447 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:57.552796 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:57.552797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:57.552797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:57.552855 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:57.553020 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:57.553021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:57.553021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:57.553174 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:57.553255 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:57.553412 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 06:35:57.553476 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 06:35:57.553965 2264 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56682
I20260806 06:35:57.554730 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 2360
I20260806 06:35:57.566568 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 2546
I20260806 06:35:57.586541 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 2732
I20260806 06:35:57.597195 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 2216
2026-08-06T06:35:57Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2152 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:57 dist-test-slave-3kk6 krb5kdc[2919](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:57 dist-test-slave-3kk6 krb5kdc[2919](info): set up 2 sockets
Aug 06 06:35:57 dist-test-slave-3kk6 krb5kdc[2919](info): commencing operation
krb5kdc: starting...
W20260806 06:35:58.850975 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:35:58 dist-test-slave-3kk6 krb5kdc[2919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:35:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:35:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:35:58.983582 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:35173
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:35173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:59.075099 2937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:59.075346 2937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:59.075371 2937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:59.076576 2937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:35:59.076629 2937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:59.076644 2937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:59.076658 2937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:35:59.076718 2937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:35:59.078331 2937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:35173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:35173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:59.078612 2937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:59.078917 2937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:59.084122 2945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:59.084100 2948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:59.084275 2943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:59.084509 2937 server_base.cc:1034] running on GCE node
I20260806 06:35:59.084844 2937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:59.085228 2937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:59.086429 2937 hybrid_clock.cc:648] HybridClock initialized: now 1785998159086406 us; error 67 us; skew 500 ppm
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[2919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998159, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:59.094105 2937 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:35:59.094833 2937 webserver.cc:460] Webserver started at http://127.29.142.190:33847/ using document root <none> and password file <none>
I20260806 06:35:59.095116 2937 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:59.095177 2937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:59.095294 2937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:59.096441 2937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "d5a54d84f0064212ac4f046b92ae7726"
format_stamp: "Formatted at 2026-08-06 06:35:59 on dist-test-slave-3kk6"
server_key: "8438c9fd409ecf6815a4745d8921fe15"
server_key_iv: "ff333cdc352620391f2c960b8251d03b"
server_key_version: "encryptionkey@0"
I20260806 06:35:59.096778 2937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "d5a54d84f0064212ac4f046b92ae7726"
format_stamp: "Formatted at 2026-08-06 06:35:59 on dist-test-slave-3kk6"
server_key: "8438c9fd409ecf6815a4745d8921fe15"
server_key_iv: "ff333cdc352620391f2c960b8251d03b"
server_key_version: "encryptionkey@0"
I20260806 06:35:59.098217 2937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:35:59.099071 2959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:59.099267 2937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:59.099376 2937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "d5a54d84f0064212ac4f046b92ae7726"
format_stamp: "Formatted at 2026-08-06 06:35:59 on dist-test-slave-3kk6"
server_key: "8438c9fd409ecf6815a4745d8921fe15"
server_key_iv: "ff333cdc352620391f2c960b8251d03b"
server_key_version: "encryptionkey@0"
I20260806 06:35:59.099448 2937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:59.111827 2937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:59.114605 2937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:59.114849 2937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:59.119637 2937 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:35173
I20260806 06:35:59.119675 3040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:35173 every 8 connection(s)
I20260806 06:35:59.120296 2937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:35:59.121114 3041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:35:59.123653 3041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726: Bootstrap starting.
I20260806 06:35:59.124318 3041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726: Neither blocks nor log segments found. Creating new log.
I20260806 06:35:59.124634 3041 log.cc:824] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726: Log is configured to *not* fsync() on all Append() calls
I20260806 06:35:59.125589 3041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726: No bootstrap required, opened a new log
I20260806 06:35:59.127338 3041 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a54d84f0064212ac4f046b92ae7726" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35173 } }
I20260806 06:35:59.127519 3041 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:35:59.127570 3041 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a54d84f0064212ac4f046b92ae7726, State: Initialized, Role: FOLLOWER
I20260806 06:35:59.127787 3041 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a54d84f0064212ac4f046b92ae7726" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35173 } }
I20260806 06:35:59.127902 3041 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:35:59.127944 3041 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:35:59.128001 3041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:35:59.129048 3041 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a54d84f0064212ac4f046b92ae7726" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35173 } }
I20260806 06:35:59.129187 3041 leader_election.cc:302] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5a54d84f0064212ac4f046b92ae7726; no voters:
I20260806 06:35:59.129410 3041 leader_election.cc:288] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:35:59.129467 3045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:35:59.129806 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 2937
I20260806 06:35:59.129940 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:35:59.130137 30266 external_mini_cluster.cc:1428] Setting key ae12e3d76ab4e5423f8e5e77a30bd43f
I20260806 06:35:59.129725 3041 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:35:59.130416 3045 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [term 1 LEADER]: Becoming Leader. State: Replica: d5a54d84f0064212ac4f046b92ae7726, State: Running, Role: LEADER
I20260806 06:35:59.130545 3045 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a54d84f0064212ac4f046b92ae7726" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35173 } }
I20260806 06:35:59.131980 3049 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5a54d84f0064212ac4f046b92ae7726. Latest consensus state: current_term: 1 leader_uuid: "d5a54d84f0064212ac4f046b92ae7726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a54d84f0064212ac4f046b92ae7726" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35173 } } }
I20260806 06:35:59.132057 3047 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5a54d84f0064212ac4f046b92ae7726" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a54d84f0064212ac4f046b92ae7726" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35173 } } }
I20260806 06:35:59.132156 3047 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [sys.catalog]: This master's current role is: LEADER
I20260806 06:35:59.132092 3049 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726 [sys.catalog]: This master's current role is: LEADER
W20260806 06:35:59.132453 3069 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:35:59.132516 3069 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:35:59.132611 3072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:35:59.132817 3072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:35:59.134599 3072 catalog_manager.cc:1347] Generated new cluster ID: 3abfbc98860b456e9660fad4edccb80b
I20260806 06:35:59.134680 3072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[2919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:59.143225 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:59.130936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40574 (local address 127.29.142.190:35173)
0806 06:35:59.131127 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:59.131138 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:59.131173 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:35:59.131311 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:59.131320 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:59.131383 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:59.131553 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:59.132474 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.133816 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.134955 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.135618 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.135715 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:59.138858 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:59.138863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:59.138864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:59.138878 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:59.141077 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:59.141512 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:59.141516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:59.141516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:59.141632 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:59.141810 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:59.141811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:59.141811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:59.142125 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:59.142295 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:59.142808 (+ 513us) server_negotiation.cc:299] Negotiation successful
0806 06:35:59.142955 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260806 06:35:59.146332 3072 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:35:59.147034 3072 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:35:59.151505 3072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5a54d84f0064212ac4f046b92ae7726: Generated new TSK 0
I20260806 06:35:59.151751 3072 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:35:59.181736 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35173
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:59.272420 3081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:59.272655 3081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:59.272679 3081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:59.273893 3081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:59.273952 3081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:59.273981 3081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:35:59.275856 3081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:59.276167 3081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:59.276449 3081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:59.282290 3091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:59.282315 3087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:59.282290 3088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:59.282945 3081 server_base.cc:1034] running on GCE node
I20260806 06:35:59.283228 3081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:59.283511 3081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:59.284718 3081 hybrid_clock.cc:648] HybridClock initialized: now 1785998159284698 us; error 41 us; skew 500 ppm
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[2919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998159, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:59.287909 3081 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:35:59.288687 3081 webserver.cc:460] Webserver started at http://127.29.142.129:38971/ using document root <none> and password file <none>
I20260806 06:35:59.289036 3081 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:59.289119 3081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:59.289294 3081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:59.290467 3081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "5f19ba36b5764866b7961b4f234fca57"
format_stamp: "Formatted at 2026-08-06 06:35:59 on dist-test-slave-3kk6"
server_key: "798ff6ec17dcc446dc9aff4f7960cf37"
server_key_iv: "a1aa58e7cc76e62e0ab1825f0f361502"
server_key_version: "encryptionkey@0"
I20260806 06:35:59.290908 3081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "5f19ba36b5764866b7961b4f234fca57"
format_stamp: "Formatted at 2026-08-06 06:35:59 on dist-test-slave-3kk6"
server_key: "798ff6ec17dcc446dc9aff4f7960cf37"
server_key_iv: "a1aa58e7cc76e62e0ab1825f0f361502"
server_key_version: "encryptionkey@0"
I20260806 06:35:59.292465 3081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:59.293308 3101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:59.293540 3081 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:59.293644 3081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "5f19ba36b5764866b7961b4f234fca57"
format_stamp: "Formatted at 2026-08-06 06:35:59 on dist-test-slave-3kk6"
server_key: "798ff6ec17dcc446dc9aff4f7960cf37"
server_key_iv: "a1aa58e7cc76e62e0ab1825f0f361502"
server_key_version: "encryptionkey@0"
I20260806 06:35:59.293730 3081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:59.338423 3081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:59.340899 3081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:59.341071 3081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:59.341364 3081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:59.341727 3081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:59.341784 3081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:59.341822 3081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:59.341874 3081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:59.348208 3081 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:45979
I20260806 06:35:59.348271 3267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:45979 every 8 connection(s)
I20260806 06:35:59.348649 3081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[2919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998159, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:59.358784 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:59.349512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:36185 (local address 127.29.142.190:35173)
0806 06:35:59.349583 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:59.349586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:59.349794 (+ 208us) server_negotiation.cc:407] Connection header received
0806 06:35:59.350847 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:59.350856 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:59.350875 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:59.350942 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:59.351668 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.352203 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.353256 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.353488 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.353572 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:59.355941 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:59.355947 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:59.355947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:59.355957 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:59.357259 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:59.357680 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:59.357681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:59.357682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:59.357744 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:59.358023 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:59.358024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:59.358024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:59.358145 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:59.358221 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:59.358544 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 06:35:59.358685 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:35:59.358783 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 3081
I20260806 06:35:59.358916 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:35:59.358954 3270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:59.349489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:35173 (local address 127.29.142.129:36185)
0806 06:35:59.349659 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:59.349699 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:59.350722 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:59.351021 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:59.351024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:59.351301 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:59.351541 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:59.351546 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.352318 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.352319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:59.353147 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:59.353150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.353582 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.353584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:59.353726 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:59.354362 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:59.354371 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:59.355821 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:59.357348 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:59.357364 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:59.357366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:59.357597 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:59.357820 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:59.357820 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:59.357821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:59.357944 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:59.358223 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:59.358230 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:59.358381 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 06:35:59.358496 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 06:35:59.358703 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260806 06:35:59.359154 30266 external_mini_cluster.cc:1428] Setting key 53a5dcc63df6ee6cf6b0d565534ae51d
I20260806 06:35:59.359537 3268 heartbeater.cc:347] Connected to a master server at 127.29.142.190:35173
I20260806 06:35:59.359670 3268 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:59.359877 3268 heartbeater.cc:511] Master 127.29.142.190:35173 requested a full tablet report, sending...
I20260806 06:35:59.360459 2985 ts_manager.cc:194] Registered new tserver with Master: 5f19ba36b5764866b7961b4f234fca57 (127.29.142.129:45979)
I20260806 06:35:59.361084 2985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:36185
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:35:59.387961 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35173
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:59.475356 3275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:59.475673 3275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:59.475719 3275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:59.477031 3275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:59.477111 3275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:59.477160 3275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:35:59.478966 3275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:59.479250 3275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:59.479502 3275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:59.485010 3290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:59.485010 3287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:59.485234 3275 server_base.cc:1034] running on GCE node
W20260806 06:35:59.485010 3286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:59.485560 3275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:59.485855 3275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:59.487035 3275 hybrid_clock.cc:648] HybridClock initialized: now 1785998159487014 us; error 41 us; skew 500 ppm
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[2919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998159, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:59.489851 3275 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:35:59.490346 3275 webserver.cc:460] Webserver started at http://127.29.142.130:35315/ using document root <none> and password file <none>
I20260806 06:35:59.490590 3275 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:59.490679 3275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:59.490851 3275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:59.492017 3275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "228fad056b24459fafe2e261861fafbc"
format_stamp: "Formatted at 2026-08-06 06:35:59 on dist-test-slave-3kk6"
server_key: "e6a600920328920de70e34f16577c9e7"
server_key_iv: "a6b937d37499cab7fc71abd5ee7493ce"
server_key_version: "encryptionkey@0"
I20260806 06:35:59.492415 3275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "228fad056b24459fafe2e261861fafbc"
format_stamp: "Formatted at 2026-08-06 06:35:59 on dist-test-slave-3kk6"
server_key: "e6a600920328920de70e34f16577c9e7"
server_key_iv: "a6b937d37499cab7fc71abd5ee7493ce"
server_key_version: "encryptionkey@0"
I20260806 06:35:59.493865 3275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:35:59.494640 3304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:59.494894 3275 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:35:59.495002 3275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "228fad056b24459fafe2e261861fafbc"
format_stamp: "Formatted at 2026-08-06 06:35:59 on dist-test-slave-3kk6"
server_key: "e6a600920328920de70e34f16577c9e7"
server_key_iv: "a6b937d37499cab7fc71abd5ee7493ce"
server_key_version: "encryptionkey@0"
I20260806 06:35:59.495088 3275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:59.505126 3275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:59.507581 3275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:59.507767 3275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:59.508051 3275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:59.508401 3275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:59.508457 3275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:59.508495 3275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:59.508541 3275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:59.514776 3275 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:45025
I20260806 06:35:59.514870 3468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:45025 every 8 connection(s)
I20260806 06:35:59.515239 3275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[2919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998159, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:59.524315 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 3275
I20260806 06:35:59.524468 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:35:59.524674 30266 external_mini_cluster.cc:1428] Setting key cc8c2ab82902b827cd241edb4f5de3cd
I20260806 06:35:59.525811 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:59.516113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:36525 (local address 127.29.142.190:35173)
0806 06:35:59.516218 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:59.516221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:59.516416 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:35:59.517492 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:59.517493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:59.517507 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:59.517563 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:59.518268 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.518857 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.519911 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.520117 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.520220 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:59.522602 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:59.522603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:59.522603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:59.522611 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:59.523989 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:59.524493 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:59.524494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:59.524495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:59.524596 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:59.524990 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:59.524991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:59.524991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:59.525138 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:59.525201 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:59.525566 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 06:35:59.525669 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:35:59.525895 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:59.516061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:35173 (local address 127.29.142.130:36525)
0806 06:35:59.516276 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:59.516327 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:59.517367 (+ 1040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:59.517652 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:59.517657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:59.517956 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:59.518158 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:59.518163 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.518967 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.518968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:59.519813 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:59.519815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.520242 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.520243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:59.520374 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:59.520943 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:59.520953 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:59.522499 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:59.524086 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:59.524093 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:59.524095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:59.524356 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:59.524721 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:59.524722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:59.524722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:59.524898 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:59.525213 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:59.525220 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:59.525400 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 06:35:59.525528 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 06:35:59.525683 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":112,"threads_started":1}
I20260806 06:35:59.526386 3469 heartbeater.cc:347] Connected to a master server at 127.29.142.190:35173
I20260806 06:35:59.526512 3469 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:59.526770 3469 heartbeater.cc:511] Master 127.29.142.190:35173 requested a full tablet report, sending...
I20260806 06:35:59.527241 2985 ts_manager.cc:194] Registered new tserver with Master: 228fad056b24459fafe2e261861fafbc (127.29.142.130:45025)
I20260806 06:35:59.527643 2985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:36525
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:35:59.553654 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35173
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:35:59.642149 3476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:35:59.642405 3476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:35:59.642432 3476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:35:59.643989 3476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:35:59.644054 3476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:35:59.644085 3476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:35:59.645745 3476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:35:59.646034 3476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:35:59.646301 3476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:35:59.651909 3484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:35:59.651945 3489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:59.652069 3476 server_base.cc:1034] running on GCE node
W20260806 06:35:59.651925 3485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:35:59.652593 3476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:35:59.652889 3476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:35:59.654074 3476 hybrid_clock.cc:648] HybridClock initialized: now 1785998159654057 us; error 53 us; skew 500 ppm
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[2919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998159, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:35:59.656900 3476 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:35:59.657398 3476 webserver.cc:460] Webserver started at http://127.29.142.131:46729/ using document root <none> and password file <none>
I20260806 06:35:59.657670 3476 fs_manager.cc:359] Metadata directory not provided
I20260806 06:35:59.657749 3476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:35:59.657913 3476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:35:59.659129 3476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "1e510335cd334fdb810fa571a366d42a"
format_stamp: "Formatted at 2026-08-06 06:35:59 on dist-test-slave-3kk6"
server_key: "df1061841326155a086112aaa2d5213d"
server_key_iv: "098ca454fae53e8c9c8b61e6847f956c"
server_key_version: "encryptionkey@0"
I20260806 06:35:59.659592 3476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "1e510335cd334fdb810fa571a366d42a"
format_stamp: "Formatted at 2026-08-06 06:35:59 on dist-test-slave-3kk6"
server_key: "df1061841326155a086112aaa2d5213d"
server_key_iv: "098ca454fae53e8c9c8b61e6847f956c"
server_key_version: "encryptionkey@0"
I20260806 06:35:59.661079 3476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:35:59.661969 3496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:59.662194 3476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:35:59.662319 3476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "1e510335cd334fdb810fa571a366d42a"
format_stamp: "Formatted at 2026-08-06 06:35:59 on dist-test-slave-3kk6"
server_key: "df1061841326155a086112aaa2d5213d"
server_key_iv: "098ca454fae53e8c9c8b61e6847f956c"
server_key_version: "encryptionkey@0"
I20260806 06:35:59.662407 3476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:35:59.670234 3476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:35:59.672760 3476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:35:59.672955 3476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:35:59.673231 3476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:35:59.673593 3476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:35:59.673652 3476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:59.673694 3476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:35:59.673724 3476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:35:59.680035 3476 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:34785
I20260806 06:35:59.680085 3659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:34785 every 8 connection(s)
I20260806 06:35:59.680469 3476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[2919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998159, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:59.690110 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 3476
I20260806 06:35:59.690224 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:35:59.690459 30266 external_mini_cluster.cc:1428] Setting key f53a4bae390c3f70224b388088ff0b17
I20260806 06:35:59.691450 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:59.681375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:52103 (local address 127.29.142.190:35173)
0806 06:35:59.681454 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:59.681456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:59.681668 (+ 212us) server_negotiation.cc:407] Connection header received
0806 06:35:59.682782 (+ 1114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:59.682783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:59.682795 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:59.682854 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:59.683488 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.683989 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.685092 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.685313 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.685377 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:59.687859 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:59.687860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:59.687861 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:59.687872 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:59.689686 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:59.690118 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:59.690119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:59.690120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:59.690206 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:59.690557 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:59.690559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:59.690559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:59.690785 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:59.690850 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:59.691256 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 06:35:59.691359 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:35:59.691718 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:59.681370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:35173 (local address 127.29.142.131:52103)
0806 06:35:59.681544 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:35:59.681583 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:35:59.682578 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:35:59.682917 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:35:59.682920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:35:59.683202 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:35:59.683407 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:59.683411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.684135 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.684136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:59.684984 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:35:59.684986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.685401 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.685402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:35:59.685517 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:59.686179 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:35:59.686189 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:35:59.687750 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:35:59.689771 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:59.689779 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:59.689781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:59.690032 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:59.690304 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:35:59.690305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:35:59.690305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:35:59.690467 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:35:59.690846 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:35:59.690854 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:35:59.691081 (+ 227us) client_negotiation.cc:769] Sending connection context
0806 06:35:59.691221 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 06:35:59.691416 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260806 06:35:59.692216 3661 heartbeater.cc:347] Connected to a master server at 127.29.142.190:35173
I20260806 06:35:59.692337 3661 heartbeater.cc:464] Registering TS with master...
I20260806 06:35:59.692559 3661 heartbeater.cc:511] Master 127.29.142.190:35173 requested a full tablet report, sending...
I20260806 06:35:59.693022 2985 ts_manager.cc:194] Registered new tserver with Master: 1e510335cd334fdb810fa571a366d42a (127.29.142.131:34785)
I20260806 06:35:59.693385 2985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:52103
I20260806 06:35:59.702980 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[2919](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[2919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.129@KRBTEST.COM
I20260806 06:35:59.728964 3671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:59.720152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.29.142.129:45979)
0806 06:35:59.720354 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:59.720360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:59.720373 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:35:59.720404 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:59.720406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:59.720421 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:59.720497 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:59.721214 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.721971 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.722898 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.723286 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.723360 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:59.725942 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:59.725954 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:59.725954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:59.725963 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:59.727675 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:59.728071 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:59.728075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:59.728076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:59.728166 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:59.728398 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:59.728399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:59.728399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:59.728576 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:59.728707 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:59.728780 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 06:35:59.728858 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[2919](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:35:59.738441 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:35:59.730594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40580 (local address 127.29.142.190:35173)
0806 06:35:59.730643 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:35:59.730668 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 06:35:59.730722 (+ 54us) server_negotiation.cc:407] Connection header received
0806 06:35:59.730783 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:35:59.730784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:35:59.730795 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:35:59.730845 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:35:59.731366 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.732053 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.732722 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:35:59.732922 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:35:59.732980 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:35:59.735484 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:35:59.735486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:35:59.735486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:35:59.735496 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:35:59.736963 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:59.737385 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:59.737387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:59.737388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:59.737458 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:35:59.737680 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:35:59.737681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:35:59.737682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:35:59.737947 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:35:59.738072 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 06:35:59.738222 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 06:35:59.738319 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 06:35:59.738811 2985 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40580
I20260806 06:35:59.739270 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 3081
I20260806 06:35:59.751307 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 3275
I20260806 06:35:59.760372 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 3476
I20260806 06:35:59.776054 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 2937
2026-08-06T06:35:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2180 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[3679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[3679](info): set up 2 sockets
Aug 06 06:35:59 dist-test-slave-3kk6 krb5kdc[3679](info): commencing operation
krb5kdc: starting...
W20260806 06:36:01.008805 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:36:01 dist-test-slave-3kk6 krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:36:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:36:01.141374 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:45855
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:45855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:01.228441 3695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:01.228749 3695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:01.228794 3695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:01.230028 3695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:01.230105 3695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:01.230130 3695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:01.230155 3695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:01.230213 3695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:01.232064 3695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:45855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:45855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:01.232388 3695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:01.232631 3695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:01.237831 3702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:01.237823 3700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:01.237823 3704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:01.238332 3695 server_base.cc:1034] running on GCE node
I20260806 06:36:01.238579 3695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:01.238945 3695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:01.240101 3695 hybrid_clock.cc:648] HybridClock initialized: now 1785998161240077 us; error 42 us; skew 500 ppm
Aug 06 06:36:01 dist-test-slave-3kk6 krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998161, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:01.242866 3695 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:36:01.243317 3695 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:01.243395 3695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:01.243556 3695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:01.244721 3695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "bcd76727b7564a1fb42786807c5e56d8"
format_stamp: "Formatted at 2026-08-06 06:36:01 on dist-test-slave-3kk6"
server_key: "ca22b926b0d9b7221fe4f17d602b9d9f"
server_key_iv: "9c0b8257a016959e9f86e1ea2f0f4ed8"
server_key_version: "encryptionkey@0"
I20260806 06:36:01.245113 3695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "bcd76727b7564a1fb42786807c5e56d8"
format_stamp: "Formatted at 2026-08-06 06:36:01 on dist-test-slave-3kk6"
server_key: "ca22b926b0d9b7221fe4f17d602b9d9f"
server_key_iv: "9c0b8257a016959e9f86e1ea2f0f4ed8"
server_key_version: "encryptionkey@0"
I20260806 06:36:01.246572 3695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:01.247509 3714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:01.247707 3695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:01.247822 3695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "bcd76727b7564a1fb42786807c5e56d8"
format_stamp: "Formatted at 2026-08-06 06:36:01 on dist-test-slave-3kk6"
server_key: "ca22b926b0d9b7221fe4f17d602b9d9f"
server_key_iv: "9c0b8257a016959e9f86e1ea2f0f4ed8"
server_key_version: "encryptionkey@0"
I20260806 06:36:01.247907 3695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:01.291926 3695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:01.294598 3695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:01.294817 3695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:01.299235 3695 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:45855
I20260806 06:36:01.299268 3795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:45855 every 8 connection(s)
I20260806 06:36:01.299609 3695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:01.300768 3796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:01.303347 3796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8: Bootstrap starting.
I20260806 06:36:01.303979 3796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:01.304337 3796 log.cc:824] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:01.305083 3796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8: No bootstrap required, opened a new log
I20260806 06:36:01.306958 3796 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcd76727b7564a1fb42786807c5e56d8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45855 } }
I20260806 06:36:01.307123 3796 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:01.307148 3796 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcd76727b7564a1fb42786807c5e56d8, State: Initialized, Role: FOLLOWER
I20260806 06:36:01.307260 3796 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcd76727b7564a1fb42786807c5e56d8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45855 } }
I20260806 06:36:01.307317 3796 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:01.307338 3796 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:01.307415 3796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:01.308064 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 3695
I20260806 06:36:01.308187 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:01.308386 30266 external_mini_cluster.cc:1428] Setting key e008930c9af39d0835cedb574a01b7b5
I20260806 06:36:01.308426 3796 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcd76727b7564a1fb42786807c5e56d8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45855 } }
I20260806 06:36:01.308586 3796 leader_election.cc:302] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcd76727b7564a1fb42786807c5e56d8; no voters:
I20260806 06:36:01.308789 3796 leader_election.cc:288] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:01.308934 3800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:01.309067 3800 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [term 1 LEADER]: Becoming Leader. State: Replica: bcd76727b7564a1fb42786807c5e56d8, State: Running, Role: LEADER
I20260806 06:36:01.309100 3796 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:01.309176 3800 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcd76727b7564a1fb42786807c5e56d8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45855 } }
I20260806 06:36:01.309513 3800 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcd76727b7564a1fb42786807c5e56d8. Latest consensus state: current_term: 1 leader_uuid: "bcd76727b7564a1fb42786807c5e56d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcd76727b7564a1fb42786807c5e56d8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45855 } } }
I20260806 06:36:01.309525 3801 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcd76727b7564a1fb42786807c5e56d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcd76727b7564a1fb42786807c5e56d8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45855 } } }
I20260806 06:36:01.309616 3800 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:01.309653 3801 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:01.310509 3811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:01.311141 3811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:01.313580 3811 catalog_manager.cc:1347] Generated new cluster ID: 39bb391011574ab18d2ec5898277b59b
I20260806 06:36:01.313642 3811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:36:01 dist-test-slave-3kk6 krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:01.320205 3806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.308954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42524 (local address 127.29.142.190:45855)
0806 06:36:01.309278 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:01.309289 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:01.309340 (+ 51us) server_negotiation.cc:407] Connection header received
0806 06:36:01.309464 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:01.309472 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:01.309536 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:01.310622 (+ 1086us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:01.310842 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.312119 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.313206 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.313789 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.313945 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.316558 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:01.316563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:01.316568 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:01.316581 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:01.318403 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:01.318909 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:01.318913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:01.318913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:01.319051 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:01.319290 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:01.319291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:01.319291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:01.319569 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:01.319766 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:01.319840 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:36:01.319998 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":95,"threads_started":1}
I20260806 06:36:01.323299 3811 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:01.324043 3811 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:01.334081 3811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8: Generated new TSK 0
I20260806 06:36:01.334345 3811 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:36:01.374205 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:45855
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:01.493834 3836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:01.494110 3836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:01.494156 3836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:01.495395 3836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:01.495481 3836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:01.495525 3836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:36:01.497150 3836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:45855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:01.497447 3836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:01.497720 3836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:01.503134 3845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:01.503136 3848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:01.503136 3846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:01.503439 3836 server_base.cc:1034] running on GCE node
I20260806 06:36:01.503746 3836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:01.504041 3836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:01.505229 3836 hybrid_clock.cc:648] HybridClock initialized: now 1785998161505209 us; error 56 us; skew 500 ppm
Aug 06 06:36:01 dist-test-slave-3kk6 krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998161, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:01.508239 3836 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:36:01.508673 3836 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:01.508733 3836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:01.508908 3836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:01.510221 3836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "d46009eeaf26459294ffddb9d23f88b4"
format_stamp: "Formatted at 2026-08-06 06:36:01 on dist-test-slave-3kk6"
server_key: "4fc8c31e3cced154e975e79dbcad9802"
server_key_iv: "565985b254fbe025c5702404313405ac"
server_key_version: "encryptionkey@0"
I20260806 06:36:01.510627 3836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "d46009eeaf26459294ffddb9d23f88b4"
format_stamp: "Formatted at 2026-08-06 06:36:01 on dist-test-slave-3kk6"
server_key: "4fc8c31e3cced154e975e79dbcad9802"
server_key_iv: "565985b254fbe025c5702404313405ac"
server_key_version: "encryptionkey@0"
I20260806 06:36:01.512102 3836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:01.512857 3865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:01.513043 3836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:01.513160 3836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "d46009eeaf26459294ffddb9d23f88b4"
format_stamp: "Formatted at 2026-08-06 06:36:01 on dist-test-slave-3kk6"
server_key: "4fc8c31e3cced154e975e79dbcad9802"
server_key_iv: "565985b254fbe025c5702404313405ac"
server_key_version: "encryptionkey@0"
I20260806 06:36:01.513247 3836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:01.530441 3836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:01.533313 3836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:01.533511 3836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:01.533800 3836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:01.534155 3836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:01.534214 3836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:01.534255 3836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:01.534307 3836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:01.540480 3836 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:43047
I20260806 06:36:01.540518 4022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:43047 every 8 connection(s)
I20260806 06:36:01.540860 3836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:36:01.541447 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 3836
I20260806 06:36:01.541574 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:36:01.541827 30266 external_mini_cluster.cc:1428] Setting key 65e2e93416e4fb7ec35fcdb79687b228
Aug 06 06:36:01 dist-test-slave-3kk6 krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998161, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
I20260806 06:36:01.552400 3806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.542055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:52655 (local address 127.29.142.190:45855)
0806 06:36:01.542133 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:01.542135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:01.542147 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:36:01.542710 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:01.542712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:01.542729 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:01.542777 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:01.543829 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.544358 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.545419 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.545656 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.545742 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.548806 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:01.548809 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:01.548809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:01.548824 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:01.550723 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:01.551210 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:01.551212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:01.551212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:01.551284 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:01.551563 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:01.551564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:01.551564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:01.551749 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:01.551825 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:01.552151 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 06:36:01.552255 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:36:01.552515 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.541766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:45855 (local address 127.29.142.129:52655)
0806 06:36:01.541948 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:01.541993 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:01.542553 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:01.542883 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:01.542887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:01.543198 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:01.543700 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:01.543706 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.544465 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.544467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:01.545312 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:01.545315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.545752 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.545753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:01.545900 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.546598 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:01.546629 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:01.548666 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:01.550820 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:01.550833 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:01.550836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:01.551104 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:01.551355 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:01.551356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:01.551356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:01.551483 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:01.551821 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:01.551828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:01.551971 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 06:36:01.552106 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 06:36:01.552278 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260806 06:36:01.553090 4024 heartbeater.cc:347] Connected to a master server at 127.29.142.190:45855
I20260806 06:36:01.553226 4024 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:01.553372 4024 heartbeater.cc:511] Master 127.29.142.190:45855 requested a full tablet report, sending...
I20260806 06:36:01.553947 3749 ts_manager.cc:194] Registered new tserver with Master: d46009eeaf26459294ffddb9d23f88b4 (127.29.142.129:43047)
I20260806 06:36:01.554719 3749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:52655
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:36:01.573377 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:45855
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:01.662266 4032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:01.662515 4032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:01.662542 4032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:01.663806 4032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:01.663868 4032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:01.663901 4032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:36:01.665455 4032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:45855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:01.665676 4032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:01.665890 4032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:01.671268 4040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:01.671270 4046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:01.671270 4044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:01.671499 4032 server_base.cc:1034] running on GCE node
I20260806 06:36:01.671864 4032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:01.672106 4032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:01.673252 4032 hybrid_clock.cc:648] HybridClock initialized: now 1785998161673236 us; error 44 us; skew 500 ppm
Aug 06 06:36:01 dist-test-slave-3kk6 krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998161, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:01.676085 4032 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:36:01.676491 4032 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:01.676544 4032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:01.676646 4032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:01.677753 4032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98"
format_stamp: "Formatted at 2026-08-06 06:36:01 on dist-test-slave-3kk6"
server_key: "b28ce70210977e6973f1d837566c0a33"
server_key_iv: "e64ec8109321d7cf50860917e71ea169"
server_key_version: "encryptionkey@0"
I20260806 06:36:01.678087 4032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98"
format_stamp: "Formatted at 2026-08-06 06:36:01 on dist-test-slave-3kk6"
server_key: "b28ce70210977e6973f1d837566c0a33"
server_key_iv: "e64ec8109321d7cf50860917e71ea169"
server_key_version: "encryptionkey@0"
I20260806 06:36:01.679611 4032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:01.680387 4054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:01.680572 4032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:01.680686 4032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98"
format_stamp: "Formatted at 2026-08-06 06:36:01 on dist-test-slave-3kk6"
server_key: "b28ce70210977e6973f1d837566c0a33"
server_key_iv: "e64ec8109321d7cf50860917e71ea169"
server_key_version: "encryptionkey@0"
I20260806 06:36:01.680773 4032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:01.697556 4032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:01.700093 4032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:01.700287 4032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:01.700551 4032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:01.700899 4032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:01.700935 4032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:01.700960 4032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:01.701010 4032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:01.707221 4032 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:44841
I20260806 06:36:01.707309 4231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:44841 every 8 connection(s)
I20260806 06:36:01.707623 4032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:36:01.709865 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 4032
I20260806 06:36:01.709975 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:36:01.710215 30266 external_mini_cluster.cc:1428] Setting key 98a6cd283abd544359dbf21d7c462019
Aug 06 06:36:01 dist-test-slave-3kk6 krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998161, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:01.719374 3806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.708522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:41161 (local address 127.29.142.190:45855)
0806 06:36:01.708589 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:01.708591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:01.708887 (+ 296us) server_negotiation.cc:407] Connection header received
0806 06:36:01.709567 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:01.709568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:01.709581 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:01.709671 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:01.711101 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.711886 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.712931 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.713257 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.713345 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.716200 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:01.716202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:01.716207 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:01.716214 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:01.717755 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:01.718239 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:01.718240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:01.718241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:01.718305 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:01.718580 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:01.718581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:01.718581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:01.718782 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:01.718859 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:01.719173 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 06:36:01.719272 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:36:01.719599 4235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.708513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:45855 (local address 127.29.142.130:41161)
0806 06:36:01.708738 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:01.708793 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:01.709455 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:01.709770 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:01.709774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:01.710175 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:01.711013 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:01.711021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.712006 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.712007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:01.712837 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:01.712840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.713373 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.713374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:01.713506 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.714161 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:01.714173 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:01.716057 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:01.717870 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:01.717879 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:01.717881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:01.718133 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:01.718385 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:01.718386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:01.718386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:01.718508 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:01.718862 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:01.718870 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:01.719025 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 06:36:01.719137 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 06:36:01.719331 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
I20260806 06:36:01.720115 4232 heartbeater.cc:347] Connected to a master server at 127.29.142.190:45855
I20260806 06:36:01.720227 4232 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:01.720420 4232 heartbeater.cc:511] Master 127.29.142.190:45855 requested a full tablet report, sending...
I20260806 06:36:01.720949 3749 ts_manager.cc:194] Registered new tserver with Master: 4d4aaef6e6a545ae9d1c90275fc0ce98 (127.29.142.130:44841)
I20260806 06:36:01.721377 3749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:41161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:36:01.739744 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:45855
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:01.826251 4244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:01.826496 4244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:01.826524 4244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:01.827785 4244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:01.827836 4244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:01.827865 4244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:36:01.829444 4244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:45855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:01.829717 4244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:01.829974 4244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:01.835331 4260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:01.835386 4252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:01.835547 4244 server_base.cc:1034] running on GCE node
W20260806 06:36:01.835613 4249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:01.835953 4244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:01.836222 4244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:01.837393 4244 hybrid_clock.cc:648] HybridClock initialized: now 1785998161837374 us; error 34 us; skew 500 ppm
Aug 06 06:36:01 dist-test-slave-3kk6 krb5kdc[3679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998161, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:01.840194 4244 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:36:01.840626 4244 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:01.840700 4244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:01.840868 4244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:01.842015 4244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "baac8fb7b4034360bc18a2292cfcf6e2"
format_stamp: "Formatted at 2026-08-06 06:36:01 on dist-test-slave-3kk6"
server_key: "de8299290e7e2afac9bf486dd32500c9"
server_key_iv: "4dea220749c0a90c2cbf5e8437dc6292"
server_key_version: "encryptionkey@0"
I20260806 06:36:01.842427 4244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "baac8fb7b4034360bc18a2292cfcf6e2"
format_stamp: "Formatted at 2026-08-06 06:36:01 on dist-test-slave-3kk6"
server_key: "de8299290e7e2afac9bf486dd32500c9"
server_key_iv: "4dea220749c0a90c2cbf5e8437dc6292"
server_key_version: "encryptionkey@0"
I20260806 06:36:01.843938 4244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:01.844758 4269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:01.844939 4244 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:01.845055 4244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "baac8fb7b4034360bc18a2292cfcf6e2"
format_stamp: "Formatted at 2026-08-06 06:36:01 on dist-test-slave-3kk6"
server_key: "de8299290e7e2afac9bf486dd32500c9"
server_key_iv: "4dea220749c0a90c2cbf5e8437dc6292"
server_key_version: "encryptionkey@0"
I20260806 06:36:01.845142 4244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:01.873540 4244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:01.875898 4244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:01.876084 4244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:01.876358 4244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:01.876703 4244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:01.876766 4244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:01.876844 4244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:01.876876 4244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:01.883260 4244 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:37653
I20260806 06:36:01.883311 4433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:37653 every 8 connection(s)
I20260806 06:36:01.883620 4244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:36:01.886418 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 4244
I20260806 06:36:01.886547 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:36:01.886799 30266 external_mini_cluster.cc:1428] Setting key f4a8b303245400d0e3956247f90f2ae3
Aug 06 06:36:01 dist-test-slave-3kk6 krb5kdc[3679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998161, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:01.893814 3806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.884439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:37693 (local address 127.29.142.190:45855)
0806 06:36:01.884496 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:01.884498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:01.884710 (+ 212us) server_negotiation.cc:407] Connection header received
0806 06:36:01.885346 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:01.885348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:01.885362 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:01.885414 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:01.886557 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.887408 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.888548 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.888739 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.888801 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.891087 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:01.891089 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:01.891089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:01.891097 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:01.892591 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:01.892980 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:01.892981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:01.892982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:01.893037 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:01.893267 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:01.893267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:01.893267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:01.893381 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:01.893433 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:01.893676 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 06:36:01.893743 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:36:01.894012 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.884381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:45855 (local address 127.29.142.131:37693)
0806 06:36:01.884564 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:01.884610 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:01.885225 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:01.885496 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:01.885499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:01.885793 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:01.886425 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:01.886433 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.887509 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.887510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:01.888452 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:01.888456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.888821 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.888822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:01.888927 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.889472 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:01.889480 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:01.890988 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:01.892672 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:01.892678 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:01.892680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:01.892911 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:01.893097 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:01.893097 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:01.893098 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:01.893210 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:01.893437 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:01.893442 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:01.893564 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 06:36:01.893648 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:36:01.893787 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
I20260806 06:36:01.894577 4434 heartbeater.cc:347] Connected to a master server at 127.29.142.190:45855
I20260806 06:36:01.894721 4434 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:01.894912 4434 heartbeater.cc:511] Master 127.29.142.190:45855 requested a full tablet report, sending...
I20260806 06:36:01.895344 3749 ts_manager.cc:194] Registered new tserver with Master: baac8fb7b4034360bc18a2292cfcf6e2 (127.29.142.131:37653)
I20260806 06:36:01.895737 3749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:37693
I20260806 06:36:01.898537 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:36:01.906213 3806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.900022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42532 (local address 127.29.142.190:45855)
0806 06:36:01.900096 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:01.900098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:01.900246 (+ 148us) server_negotiation.cc:407] Connection header received
0806 06:36:01.900268 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:01.900269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:01.900278 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:01.900315 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:01.900940 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.901432 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.902157 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.902352 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.902413 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.903782 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:01.903788 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:01.903788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:01.903796 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:01.905151 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:01.905483 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:01.905483 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:01.905484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:01.905541 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:01.905716 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:01.905716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:01.905717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:01.905877 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:01.905955 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:01.906061 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 06:36:01.906130 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:36:01.907954 3749 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:36:01.909857 3749 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:36:01.916975 4449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.912694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.29.142.130:44841)
0806 06:36:01.912922 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:01.912928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:01.912941 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:36:01.912987 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:01.912990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:01.913008 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:01.913112 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:01.913416 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.914374 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.915860 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.916560 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.916653 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.916715 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:01.916839 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 06:36:01.916880 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":112,"threads_started":1}
I20260806 06:36:01.916960 4446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.912543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:44841 (local address 127.0.0.1:33620)
0806 06:36:01.912722 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:01.912766 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:01.912857 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:01.913174 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:01.913177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:01.913179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:01.913253 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:01.913255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.914478 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.914479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:01.915768 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:01.915771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.916651 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.916652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:01.916734 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.916750 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 06:36:01.916823 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:36:01.916880 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
I20260806 06:36:01.917335 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.912543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:43047 (local address 127.0.0.1:43964)
0806 06:36:01.912997 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:01.913008 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:01.913086 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:01.913438 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:01.913438 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:01.913441 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:01.913502 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:01.913503 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.914999 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.915000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:01.916093 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:01.916096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.917077 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.917079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:01.917146 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.917151 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:36:01.917187 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 06:36:01.917230 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":7,"thread_start_us":65,"threads_started":1}
I20260806 06:36:01.917506 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.912970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.29.142.131:37653)
0806 06:36:01.913675 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:01.913681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:01.913691 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:36:01.913729 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:01.913731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:01.913749 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:01.913835 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:01.913980 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.915037 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.916302 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.916995 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.917062 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.917119 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:01.917346 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 06:36:01.917394 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":112,"threads_started":1}
I20260806 06:36:01.917692 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.912736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43964 (local address 127.29.142.129:43047)
0806 06:36:01.912922 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:01.912928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:01.913108 (+ 180us) server_negotiation.cc:407] Connection header received
0806 06:36:01.913163 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:01.913166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:01.913182 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:01.913340 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:01.913701 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.914882 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.916189 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.916916 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.917417 (+ 501us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.917487 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:01.917514 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 06:36:01.917583 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":84,"threads_started":1}
I20260806 06:36:01.917430 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:01.912759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:37653 (local address 127.0.0.1:37488)
0806 06:36:01.913185 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:01.913195 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:01.913247 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:01.913867 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:01.913868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:01.913871 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:01.913912 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:01.913913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.915145 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.915146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:01.916226 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:01.916229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:01.917166 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:01.917167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:01.917220 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:01.917223 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:36:01.917321 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 06:36:01.917362 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":73,"threads_started":1}
I20260806 06:36:01.918437 3928 tablet_service.cc:1467] Processing CreateTablet for tablet 0471a410f0cb4965abdf9802bf030258 (DEFAULT_TABLE table=test-table [id=a898e4b202ad4aedbfc1254ee865782b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:01.918709 4336 tablet_service.cc:1467] Processing CreateTablet for tablet 0471a410f0cb4965abdf9802bf030258 (DEFAULT_TABLE table=test-table [id=a898e4b202ad4aedbfc1254ee865782b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:01.918886 3928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0471a410f0cb4965abdf9802bf030258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:01.919073 4336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0471a410f0cb4965abdf9802bf030258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:01.921883 4455 tablet_bootstrap.cc:492] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2: Bootstrap starting.
I20260806 06:36:01.922122 4453 tablet_bootstrap.cc:492] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4: Bootstrap starting.
I20260806 06:36:01.922408 4132 tablet_service.cc:1467] Processing CreateTablet for tablet 0471a410f0cb4965abdf9802bf030258 (DEFAULT_TABLE table=test-table [id=a898e4b202ad4aedbfc1254ee865782b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:01.922673 4455 tablet_bootstrap.cc:654] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:01.922825 4132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0471a410f0cb4965abdf9802bf030258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:01.922964 4453 tablet_bootstrap.cc:654] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:01.923486 4455 log.cc:824] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:01.923620 4453 log.cc:824] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:01.924343 4455 tablet_bootstrap.cc:492] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2: No bootstrap required, opened a new log
I20260806 06:36:01.924418 4455 ts_tablet_manager.cc:1397] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:36:01.924683 4453 tablet_bootstrap.cc:492] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4: No bootstrap required, opened a new log
I20260806 06:36:01.924768 4453 ts_tablet_manager.cc:1397] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:36:01.925940 4455 raft_consensus.cc:348] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 44841 } } peers { permanent_uuid: "d46009eeaf26459294ffddb9d23f88b4" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 43047 } } peers { permanent_uuid: "baac8fb7b4034360bc18a2292cfcf6e2" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37653 } }
I20260806 06:36:01.926100 4455 raft_consensus.cc:374] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:01.926149 4455 raft_consensus.cc:729] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baac8fb7b4034360bc18a2292cfcf6e2, State: Initialized, Role: FOLLOWER
I20260806 06:36:01.926265 4455 consensus_queue.cc:260] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 44841 } } peers { permanent_uuid: "d46009eeaf26459294ffddb9d23f88b4" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 43047 } } peers { permanent_uuid: "baac8fb7b4034360bc18a2292cfcf6e2" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37653 } }
I20260806 06:36:01.926573 4434 heartbeater.cc:503] Master 127.29.142.190:45855 was elected leader, sending a full tablet report...
I20260806 06:36:01.926576 4455 ts_tablet_manager.cc:1428] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:36:01.927026 4453 raft_consensus.cc:348] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 44841 } } peers { permanent_uuid: "d46009eeaf26459294ffddb9d23f88b4" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 43047 } } peers { permanent_uuid: "baac8fb7b4034360bc18a2292cfcf6e2" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37653 } }
I20260806 06:36:01.927191 4453 raft_consensus.cc:374] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:01.927217 4453 raft_consensus.cc:729] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d46009eeaf26459294ffddb9d23f88b4, State: Initialized, Role: FOLLOWER
I20260806 06:36:01.927352 4453 consensus_queue.cc:260] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 44841 } } peers { permanent_uuid: "d46009eeaf26459294ffddb9d23f88b4" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 43047 } } peers { permanent_uuid: "baac8fb7b4034360bc18a2292cfcf6e2" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37653 } }
I20260806 06:36:01.927671 4453 ts_tablet_manager.cc:1428] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:36:01.927697 4024 heartbeater.cc:503] Master 127.29.142.190:45855 was elected leader, sending a full tablet report...
I20260806 06:36:01.927904 4462 tablet_bootstrap.cc:492] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98: Bootstrap starting.
I20260806 06:36:01.928709 4462 tablet_bootstrap.cc:654] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:01.929276 4462 log.cc:824] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:01.930183 4462 tablet_bootstrap.cc:492] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98: No bootstrap required, opened a new log
I20260806 06:36:01.930259 4462 ts_tablet_manager.cc:1397] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:36:01.931830 4462 raft_consensus.cc:348] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 44841 } } peers { permanent_uuid: "d46009eeaf26459294ffddb9d23f88b4" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 43047 } } peers { permanent_uuid: "baac8fb7b4034360bc18a2292cfcf6e2" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37653 } }
I20260806 06:36:01.931975 4462 raft_consensus.cc:374] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:01.931998 4462 raft_consensus.cc:729] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4aaef6e6a545ae9d1c90275fc0ce98, State: Initialized, Role: FOLLOWER
I20260806 06:36:01.932123 4462 consensus_queue.cc:260] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 44841 } } peers { permanent_uuid: "d46009eeaf26459294ffddb9d23f88b4" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 43047 } } peers { permanent_uuid: "baac8fb7b4034360bc18a2292cfcf6e2" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37653 } }
I20260806 06:36:01.932399 4462 ts_tablet_manager.cc:1428] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:36:01.932458 4232 heartbeater.cc:503] Master 127.29.142.190:45855 was elected leader, sending a full tablet report...
W20260806 06:36:01.958449 4233 tablet.cc:2369] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:36:02.041735 4025 tablet.cc:2369] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:36:02.134234 4435 tablet.cc:2369] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:36:02.254065 4465 raft_consensus.cc:482] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:36:02.254245 4465 raft_consensus.cc:504] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 44841 } } peers { permanent_uuid: "d46009eeaf26459294ffddb9d23f88b4" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 43047 } } peers { permanent_uuid: "baac8fb7b4034360bc18a2292cfcf6e2" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37653 } }
I20260806 06:36:02.254581 4465 leader_election.cc:288] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d4aaef6e6a545ae9d1c90275fc0ce98 (127.29.142.130:44841), baac8fb7b4034360bc18a2292cfcf6e2 (127.29.142.131:37653)
I20260806 06:36:02.258453 4449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.254979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:57173 (local address 127.29.142.130:44841)
0806 06:36:02.255050 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.255052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.255096 (+ 44us) server_negotiation.cc:407] Connection header received
0806 06:36:02.255174 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.255175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.255186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.255275 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:02.255531 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:02.256420 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:02.257590 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:02.257954 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:02.258013 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:02.258020 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.258312 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.258372 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:36:02.258456 4469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.254877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:44841 (local address 127.29.142.129:57173)
0806 06:36:02.255015 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:02.255029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:02.255106 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:02.255320 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:02.255321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:02.255324 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:02.255415 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:02.255418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:02.256533 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:02.256535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:02.257480 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:02.257483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:02.258136 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:02.258137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:02.258218 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:02.258222 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:36:02.258296 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:36:02.258347 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":60,"thread_start_us":83,"threads_started":1}
I20260806 06:36:02.258692 4470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.255083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:37653 (local address 127.29.142.129:42139)
0806 06:36:02.255310 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:02.255321 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:02.255387 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:02.255530 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:02.255531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:02.255532 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:02.255600 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:02.255602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:02.256577 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:02.256578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:02.257478 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:02.257481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:02.258377 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:02.258378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:02.258452 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:02.258456 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:36:02.258523 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:36:02.258567 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":1,"thread_start_us":61,"threads_started":1}
I20260806 06:36:02.258708 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.255236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:42139 (local address 127.29.142.131:37653)
0806 06:36:02.255315 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.255317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.255370 (+ 53us) server_negotiation.cc:407] Connection header received
0806 06:36:02.255435 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.255436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.255446 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.255504 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:02.255691 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:02.256451 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:02.257584 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:02.258288 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:02.258373 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:02.258382 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.258549 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.258594 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:36:02.258922 4163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0471a410f0cb4965abdf9802bf030258" candidate_uuid: "d46009eeaf26459294ffddb9d23f88b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98" is_pre_election: true
I20260806 06:36:02.259115 4163 raft_consensus.cc:2454] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d46009eeaf26459294ffddb9d23f88b4 in term 0.
I20260806 06:36:02.259246 4365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0471a410f0cb4965abdf9802bf030258" candidate_uuid: "d46009eeaf26459294ffddb9d23f88b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baac8fb7b4034360bc18a2292cfcf6e2" is_pre_election: true
I20260806 06:36:02.259390 3872 leader_election.cc:302] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d4aaef6e6a545ae9d1c90275fc0ce98, d46009eeaf26459294ffddb9d23f88b4; no voters:
I20260806 06:36:02.259449 4365 raft_consensus.cc:2454] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d46009eeaf26459294ffddb9d23f88b4 in term 0.
I20260806 06:36:02.259544 4465 raft_consensus.cc:2781] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:36:02.259620 4465 raft_consensus.cc:482] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:36:02.259657 4465 raft_consensus.cc:3037] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:02.260403 4465 raft_consensus.cc:504] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 44841 } } peers { permanent_uuid: "d46009eeaf26459294ffddb9d23f88b4" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 43047 } } peers { permanent_uuid: "baac8fb7b4034360bc18a2292cfcf6e2" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37653 } }
I20260806 06:36:02.260548 4465 leader_election.cc:288] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [CANDIDATE]: Term 1 election: Requested vote from peers 4d4aaef6e6a545ae9d1c90275fc0ce98 (127.29.142.130:44841), baac8fb7b4034360bc18a2292cfcf6e2 (127.29.142.131:37653)
I20260806 06:36:02.260704 4163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0471a410f0cb4965abdf9802bf030258" candidate_uuid: "d46009eeaf26459294ffddb9d23f88b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98"
I20260806 06:36:02.260759 4365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0471a410f0cb4965abdf9802bf030258" candidate_uuid: "d46009eeaf26459294ffddb9d23f88b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baac8fb7b4034360bc18a2292cfcf6e2"
I20260806 06:36:02.260793 4163 raft_consensus.cc:3037] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:02.260828 4365 raft_consensus.cc:3037] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:02.261724 4163 raft_consensus.cc:2454] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d46009eeaf26459294ffddb9d23f88b4 in term 1.
I20260806 06:36:02.261744 4365 raft_consensus.cc:2454] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d46009eeaf26459294ffddb9d23f88b4 in term 1.
I20260806 06:36:02.261909 3872 leader_election.cc:302] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: baac8fb7b4034360bc18a2292cfcf6e2, d46009eeaf26459294ffddb9d23f88b4; no voters:
I20260806 06:36:02.262017 4465 raft_consensus.cc:2781] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:02.262185 4465 raft_consensus.cc:686] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [term 1 LEADER]: Becoming Leader. State: Replica: d46009eeaf26459294ffddb9d23f88b4, State: Running, Role: LEADER
I20260806 06:36:02.262319 4465 consensus_queue.cc:237] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 44841 } } peers { permanent_uuid: "d46009eeaf26459294ffddb9d23f88b4" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 43047 } } peers { permanent_uuid: "baac8fb7b4034360bc18a2292cfcf6e2" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37653 } }
I20260806 06:36:02.263242 3749 catalog_manager.cc:5495] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d46009eeaf26459294ffddb9d23f88b4 (127.29.142.129). New cstate: current_term: 1 leader_uuid: "d46009eeaf26459294ffddb9d23f88b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 44841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d46009eeaf26459294ffddb9d23f88b4" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 43047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baac8fb7b4034360bc18a2292cfcf6e2" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37653 } health_report { overall_health: UNKNOWN } } }
I20260806 06:36:02.270704 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.268515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43972 (local address 127.29.142.129:43047)
0806 06:36:02.268569 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.268571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.268578 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:36:02.268644 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.268645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.268661 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.268722 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:36:02.268822 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:02.269271 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:02.269988 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:02.270171 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:02.270228 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:02.270384 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:36:02.270477 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:36:02.270538 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.270570 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.270585 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:36:02.272414 4163 raft_consensus.cc:1261] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98 [term 1 FOLLOWER]: Refusing update from remote peer d46009eeaf26459294ffddb9d23f88b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:02.272435 4365 raft_consensus.cc:1261] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2 [term 1 FOLLOWER]: Refusing update from remote peer d46009eeaf26459294ffddb9d23f88b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:02.272769 4474 consensus_queue.cc:1035] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4aaef6e6a545ae9d1c90275fc0ce98" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 44841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:02.272917 4465 consensus_queue.cc:1035] T 0471a410f0cb4965abdf9802bf030258 P d46009eeaf26459294ffddb9d23f88b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baac8fb7b4034360bc18a2292cfcf6e2" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:02.276021 4480 mvcc.cc:205] Tried to move back new op lower bound from 7315448472665161728 to 7315448472627220480. Current Snapshot: MvccSnapshot[applied={T|T < 7315448472665161728}]
I20260806 06:36:02.276700 4477 mvcc.cc:205] Tried to move back new op lower bound from 7315448472665161728 to 7315448472627220480. Current Snapshot: MvccSnapshot[applied={T|T < 7315448472665161728 or (T in {7315448472665161728})}]
I20260806 06:36:02.278136 3744 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:02.278246 3744 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:02.279095 3744 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bcd76727b7564a1fb42786807c5e56d8: Sending DeleteTablet for 3 replicas of tablet 0471a410f0cb4965abdf9802bf030258
I20260806 06:36:02.279434 4132 tablet_service.cc:1514] Processing DeleteTablet for tablet 0471a410f0cb4965abdf9802bf030258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:02 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:33620
I20260806 06:36:02.279448 3928 tablet_service.cc:1514] Processing DeleteTablet for tablet 0471a410f0cb4965abdf9802bf030258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:02 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:43964
I20260806 06:36:02.279500 4336 tablet_service.cc:1514] Processing DeleteTablet for tablet 0471a410f0cb4965abdf9802bf030258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:02 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:37488
I20260806 06:36:02.279829 4491 tablet_replica.cc:330] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98: stopping tablet replica
I20260806 06:36:02.279984 4491 raft_consensus.cc:2229] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:02.280076 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 3836
I20260806 06:36:02.285386 4491 raft_consensus.cc:2258] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:02.285727 4493 tablet_replica.cc:330] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2: stopping tablet replica
I20260806 06:36:02.285882 4493 raft_consensus.cc:2229] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:02.285975 4491 ts_tablet_manager.cc:1905] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:02.286000 4493 raft_consensus.cc:2258] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:02.286614 4493 ts_tablet_manager.cc:1905] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:02.288412 4491 ts_tablet_manager.cc:1918] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:02.288439 4493 ts_tablet_manager.cc:1918] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:02.288509 4493 log.cc:1196] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/0471a410f0cb4965abdf9802bf030258
I20260806 06:36:02.288604 4491 log.cc:1196] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/0471a410f0cb4965abdf9802bf030258
I20260806 06:36:02.288924 4493 ts_tablet_manager.cc:1939] T 0471a410f0cb4965abdf9802bf030258 P baac8fb7b4034360bc18a2292cfcf6e2: Deleting consensus metadata
I20260806 06:36:02.289004 4491 ts_tablet_manager.cc:1939] T 0471a410f0cb4965abdf9802bf030258 P 4d4aaef6e6a545ae9d1c90275fc0ce98: Deleting consensus metadata
I20260806 06:36:02.289462 3724 catalog_manager.cc:4841] TS baac8fb7b4034360bc18a2292cfcf6e2 (127.29.142.131:37653): tablet 0471a410f0cb4965abdf9802bf030258 (table test-table [id=a898e4b202ad4aedbfc1254ee865782b]) successfully deleted
I20260806 06:36:02.289600 3724 catalog_manager.cc:4841] TS 4d4aaef6e6a545ae9d1c90275fc0ce98 (127.29.142.130:44841): tablet 0471a410f0cb4965abdf9802bf030258 (table test-table [id=a898e4b202ad4aedbfc1254ee865782b]) successfully deleted
W20260806 06:36:02.294008 3723 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:43047 (error 108)
I20260806 06:36:02.294224 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 4032
I20260806 06:36:02.298187 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.297995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:43047 (local address 127.0.0.1:43982)
0806 06:36:02.298069 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:02.298131 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:43047: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 06:36:02.299777 3723 catalog_manager.cc:4584] TS d46009eeaf26459294ffddb9d23f88b4 (127.29.142.129:43047): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0471a410f0cb4965abdf9802bf030258: Network error: Client connection negotiation failed: client connection to 127.29.142.129:43047: connect: Connection refused (error 111)
I20260806 06:36:02.307299 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 4244
I20260806 06:36:02.321640 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 3695
2026-08-06T06:36:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2547 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T06:36:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:02Z Disabled control of system clock
I20260806 06:36:02.359212 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:40745
--webserver_interface=127.29.142.190
--webserver_port=0
--builtin_ntp_servers=127.29.142.148:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:40745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:36:02.446250 4499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:02.446503 4499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:02.446529 4499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:02.447775 4499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:02.447829 4499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:02.447844 4499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:02.447856 4499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:02.447868 4499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:02.449218 4499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:40745
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:40745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:02.449440 4499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:02.449723 4499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:02.455087 4509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:02.455168 4499 server_base.cc:1034] running on GCE node
W20260806 06:36:02.455083 4511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:02.455083 4515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:02.455559 4499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:02.455844 4499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:02.457023 4499 hybrid_clock.cc:648] HybridClock initialized: now 1785998162457010 us; error 34 us; skew 500 ppm
I20260806 06:36:02.458407 4499 webserver.cc:460] Webserver started at http://127.29.142.190:40353/ using document root <none> and password file <none>
I20260806 06:36:02.458750 4499 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:02.458829 4499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:02.458990 4499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:02.460139 4499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "662ce1de341f40e7a01854587e7c38b3"
format_stamp: "Formatted at 2026-08-06 06:36:02 on dist-test-slave-3kk6"
server_key: "78a43cec74bc3e13be0cfd3d26f2d120"
server_key_iv: "4c805c79ea805b785838dcea88aa4ede"
server_key_version: "encryptionkey@0"
I20260806 06:36:02.460541 4499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "662ce1de341f40e7a01854587e7c38b3"
format_stamp: "Formatted at 2026-08-06 06:36:02 on dist-test-slave-3kk6"
server_key: "78a43cec74bc3e13be0cfd3d26f2d120"
server_key_iv: "4c805c79ea805b785838dcea88aa4ede"
server_key_version: "encryptionkey@0"
I20260806 06:36:02.461933 4499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:02.462759 4522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:02.462947 4499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:02.463042 4499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "662ce1de341f40e7a01854587e7c38b3"
format_stamp: "Formatted at 2026-08-06 06:36:02 on dist-test-slave-3kk6"
server_key: "78a43cec74bc3e13be0cfd3d26f2d120"
server_key_iv: "4c805c79ea805b785838dcea88aa4ede"
server_key_version: "encryptionkey@0"
I20260806 06:36:02.463128 4499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:02.466822 4499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:02.467123 4499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:02.467279 4499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:02.471482 4499 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:40745
I20260806 06:36:02.471520 4598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:40745 every 8 connection(s)
I20260806 06:36:02.471899 4499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:02.472607 4599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:02.474424 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 4499
I20260806 06:36:02.474527 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:02.474761 30266 external_mini_cluster.cc:1428] Setting key 528e16c65e9614399426d7170cd8fb0a
I20260806 06:36:02.476114 4599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3: Bootstrap starting.
I20260806 06:36:02.476936 4599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:02.477293 4599 log.cc:824] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:02.477793 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.475374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36818 (local address 127.29.142.190:40745)
0806 06:36:02.475613 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.475620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.475646 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:36:02.476045 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.476051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.476091 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.476265 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:02.477151 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:02.477154 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:02.477154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:02.477158 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:02.477204 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:02.477214 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:02.477274 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.477399 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.477578 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
I20260806 06:36:02.478169 4599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3: No bootstrap required, opened a new log
I20260806 06:36:02.479763 4599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "662ce1de341f40e7a01854587e7c38b3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40745 } }
I20260806 06:36:02.479915 4599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:02.479939 4599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 662ce1de341f40e7a01854587e7c38b3, State: Initialized, Role: FOLLOWER
I20260806 06:36:02.480091 4599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "662ce1de341f40e7a01854587e7c38b3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40745 } }
I20260806 06:36:02.480178 4599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:02.480232 4599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:02.480284 4599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:02.481137 4599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "662ce1de341f40e7a01854587e7c38b3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40745 } }
I20260806 06:36:02.481261 4599 leader_election.cc:302] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 662ce1de341f40e7a01854587e7c38b3; no voters:
I20260806 06:36:02.481434 4599 leader_election.cc:288] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:02.481463 4606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:02.481611 4606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [term 1 LEADER]: Becoming Leader. State: Replica: 662ce1de341f40e7a01854587e7c38b3, State: Running, Role: LEADER
I20260806 06:36:02.481747 4599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:02.481761 4606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "662ce1de341f40e7a01854587e7c38b3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40745 } }
I20260806 06:36:02.482219 4609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 662ce1de341f40e7a01854587e7c38b3. Latest consensus state: current_term: 1 leader_uuid: "662ce1de341f40e7a01854587e7c38b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "662ce1de341f40e7a01854587e7c38b3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40745 } } }
I20260806 06:36:02.482204 4608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "662ce1de341f40e7a01854587e7c38b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "662ce1de341f40e7a01854587e7c38b3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40745 } } }
I20260806 06:36:02.482328 4609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:02.482335 4608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:02.482600 4619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:02.483131 4619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:02.484807 4619 catalog_manager.cc:1347] Generated new cluster ID: 715125f6d51844988dfca9b0883b53e1
I20260806 06:36:02.484869 4619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:36:02.493309 4619 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:02.493494 4619 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:02.497422 4619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3: Generated new TSK 0
I20260806 06:36:02.497632 4619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:36:02.504066 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:40745
--builtin_ntp_servers=127.29.142.148:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:36:02.590250 4636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:02.590487 4636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:02.590512 4636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:02.591777 4636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:02.591827 4636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:02.591861 4636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:36:02.593293 4636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:40745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:02.593547 4636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:02.593910 4636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:02.599164 4646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:02.599218 4644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:02.599164 4643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:02.600114 4636 server_base.cc:1034] running on GCE node
I20260806 06:36:02.600340 4636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:02.600595 4636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:02.601765 4636 hybrid_clock.cc:648] HybridClock initialized: now 1785998162601752 us; error 32 us; skew 500 ppm
I20260806 06:36:02.603263 4636 webserver.cc:460] Webserver started at http://127.29.142.129:45521/ using document root <none> and password file <none>
I20260806 06:36:02.603530 4636 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:02.603600 4636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:02.603746 4636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:02.604990 4636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "44475be6be544371b5fe1f4dd735c74d"
format_stamp: "Formatted at 2026-08-06 06:36:02 on dist-test-slave-3kk6"
server_key: "51271ca413123d33138b044a61d20495"
server_key_iv: "1d7ec09ea7216f5b574b2be07b83c432"
server_key_version: "encryptionkey@0"
I20260806 06:36:02.605378 4636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "44475be6be544371b5fe1f4dd735c74d"
format_stamp: "Formatted at 2026-08-06 06:36:02 on dist-test-slave-3kk6"
server_key: "51271ca413123d33138b044a61d20495"
server_key_iv: "1d7ec09ea7216f5b574b2be07b83c432"
server_key_version: "encryptionkey@0"
I20260806 06:36:02.606880 4636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260806 06:36:02.607712 4658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:02.607892 4636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:02.608006 4636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "44475be6be544371b5fe1f4dd735c74d"
format_stamp: "Formatted at 2026-08-06 06:36:02 on dist-test-slave-3kk6"
server_key: "51271ca413123d33138b044a61d20495"
server_key_iv: "1d7ec09ea7216f5b574b2be07b83c432"
server_key_version: "encryptionkey@0"
I20260806 06:36:02.608088 4636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:02.611640 4636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:02.612054 4636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:02.612208 4636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:02.612470 4636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:02.612816 4636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:02.612857 4636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:02.612879 4636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:02.612921 4636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:02.619046 4636 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:46869
I20260806 06:36:02.619086 4817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:46869 every 8 connection(s)
I20260806 06:36:02.619462 4636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:36:02.619918 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 4636
I20260806 06:36:02.620033 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:36:02.620247 30266 external_mini_cluster.cc:1428] Setting key 7b0d368e3938171939a12e604bf82ebf
I20260806 06:36:02.621304 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:40745
--builtin_ntp_servers=127.29.142.148:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 06:36:02.623142 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.620341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:43037 (local address 127.29.142.190:40745)
0806 06:36:02.620391 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.620393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.620719 (+ 326us) server_negotiation.cc:407] Connection header received
0806 06:36:02.621340 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.621341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.621347 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.621396 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:02.622891 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:02.622892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:02.622893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:02.622898 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:02.622914 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:02.622918 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:02.622972 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.623056 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.623072 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:36:02.623353 4821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.620359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:40745 (local address 127.29.142.129:43037)
0806 06:36:02.620585 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:02.620629 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:02.621232 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:02.621467 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:02.621471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:02.621480 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:02.622640 (+ 1160us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:02.622640 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:02.622728 (+ 88us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:02.622789 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:02.622976 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:02.622982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:02.622985 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:36:02.623067 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:36:02.623130 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":91,"threads_started":1}
I20260806 06:36:02.624043 4818 heartbeater.cc:347] Connected to a master server at 127.29.142.190:40745
I20260806 06:36:02.624185 4818 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:02.624373 4818 heartbeater.cc:511] Master 127.29.142.190:40745 requested a full tablet report, sending...
I20260806 06:36:02.624935 4549 ts_manager.cc:194] Registered new tserver with Master: 44475be6be544371b5fe1f4dd735c74d (127.29.142.129:46869)
W20260806 06:36:02.708293 4822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:02.708529 4822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:02.708554 4822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:02.709764 4822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:02.709810 4822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:02.709837 4822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:36:02.711396 4822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:40745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:02.711676 4822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:02.712066 4822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:36:02.717434 4822 server_base.cc:1034] running on GCE node
W20260806 06:36:02.717367 4829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:02.717308 4834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:02.717283 4831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:02.717941 4822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:02.718215 4822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:02.719386 4822 hybrid_clock.cc:648] HybridClock initialized: now 1785998162719384 us; error 39 us; skew 500 ppm
I20260806 06:36:02.720731 4822 webserver.cc:460] Webserver started at http://127.29.142.130:40903/ using document root <none> and password file <none>
I20260806 06:36:02.720997 4822 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:02.721069 4822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:02.721225 4822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:02.722389 4822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "3c7f410432c14b8f9a93023364dfaf72"
format_stamp: "Formatted at 2026-08-06 06:36:02 on dist-test-slave-3kk6"
server_key: "de651a9b0fbbd147ec35c967429e9862"
server_key_iv: "cdcc54c41c202ee39b797bdf31f98f41"
server_key_version: "encryptionkey@0"
I20260806 06:36:02.722831 4822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "3c7f410432c14b8f9a93023364dfaf72"
format_stamp: "Formatted at 2026-08-06 06:36:02 on dist-test-slave-3kk6"
server_key: "de651a9b0fbbd147ec35c967429e9862"
server_key_iv: "cdcc54c41c202ee39b797bdf31f98f41"
server_key_version: "encryptionkey@0"
I20260806 06:36:02.724188 4822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:02.725091 4843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:02.725301 4822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:02.725415 4822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "3c7f410432c14b8f9a93023364dfaf72"
format_stamp: "Formatted at 2026-08-06 06:36:02 on dist-test-slave-3kk6"
server_key: "de651a9b0fbbd147ec35c967429e9862"
server_key_iv: "cdcc54c41c202ee39b797bdf31f98f41"
server_key_version: "encryptionkey@0"
I20260806 06:36:02.725499 4822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:02.729113 4822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:02.729406 4822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:02.729560 4822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:02.729779 4822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:02.730126 4822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:02.730181 4822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:02.730225 4822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:02.730253 4822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:02.736631 4822 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:35521
I20260806 06:36:02.736693 5016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:35521 every 8 connection(s)
I20260806 06:36:02.737085 4822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:36:02.737789 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 4822
I20260806 06:36:02.737913 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:36:02.738145 30266 external_mini_cluster.cc:1428] Setting key f44f30b12591fb6dc61fe34d68b4b248
I20260806 06:36:02.739333 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:40745
--builtin_ntp_servers=127.29.142.148:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 06:36:02.740602 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.737929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:48051 (local address 127.29.142.190:40745)
0806 06:36:02.737994 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.737997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.738345 (+ 348us) server_negotiation.cc:407] Connection header received
0806 06:36:02.738942 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.738943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.738950 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.739027 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:02.740310 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:02.740312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:02.740312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:02.740318 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:02.740336 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:02.740340 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:02.740412 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.740482 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.740498 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:36:02.740725 5019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.737941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:40745 (local address 127.29.142.130:48051)
0806 06:36:02.738173 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:02.738225 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:02.738825 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:02.739105 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:02.739109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:02.739118 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:02.740138 (+ 1020us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:02.740139 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:02.740168 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:02.740199 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:02.740411 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:02.740414 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:02.740416 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:36:02.740468 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:36:02.740518 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":113,"threads_started":1}
I20260806 06:36:02.741226 5017 heartbeater.cc:347] Connected to a master server at 127.29.142.190:40745
I20260806 06:36:02.741343 5017 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:02.741519 5017 heartbeater.cc:511] Master 127.29.142.190:40745 requested a full tablet report, sending...
I20260806 06:36:02.741950 4549 ts_manager.cc:194] Registered new tserver with Master: 3c7f410432c14b8f9a93023364dfaf72 (127.29.142.130:35521)
W20260806 06:36:02.826983 5020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:02.827214 5020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:02.827239 5020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:02.828584 5020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:02.828637 5020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:02.828665 5020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:36:02.830219 5020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:40745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:02.830531 5020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:02.830894 5020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:02.836668 5025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:02.836740 5020 server_base.cc:1034] running on GCE node
W20260806 06:36:02.836683 5029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:02.836673 5026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:02.837122 5020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:02.837385 5020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:02.838533 5020 hybrid_clock.cc:648] HybridClock initialized: now 1785998162838519 us; error 33 us; skew 500 ppm
I20260806 06:36:02.839883 5020 webserver.cc:460] Webserver started at http://127.29.142.131:38445/ using document root <none> and password file <none>
I20260806 06:36:02.840140 5020 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:02.840189 5020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:02.840400 5020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:02.841557 5020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "be19316564fa46c98a8fbb34d897fbb7"
format_stamp: "Formatted at 2026-08-06 06:36:02 on dist-test-slave-3kk6"
server_key: "9a5a21f42820b3ed995188bdb746a90e"
server_key_iv: "f4231c86da1328cef67965f01e286e01"
server_key_version: "encryptionkey@0"
I20260806 06:36:02.841943 5020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "be19316564fa46c98a8fbb34d897fbb7"
format_stamp: "Formatted at 2026-08-06 06:36:02 on dist-test-slave-3kk6"
server_key: "9a5a21f42820b3ed995188bdb746a90e"
server_key_iv: "f4231c86da1328cef67965f01e286e01"
server_key_version: "encryptionkey@0"
I20260806 06:36:02.843425 5020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260806 06:36:02.844190 5037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:02.844372 5020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:02.844470 5020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "be19316564fa46c98a8fbb34d897fbb7"
format_stamp: "Formatted at 2026-08-06 06:36:02 on dist-test-slave-3kk6"
server_key: "9a5a21f42820b3ed995188bdb746a90e"
server_key_iv: "f4231c86da1328cef67965f01e286e01"
server_key_version: "encryptionkey@0"
I20260806 06:36:02.844550 5020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:02.848364 5020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:02.848667 5020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:02.848822 5020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:02.849042 5020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:02.849375 5020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:02.849408 5020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:02.849453 5020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:02.849481 5020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:02.855427 5020 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:39811
I20260806 06:36:02.855517 5193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:39811 every 8 connection(s)
I20260806 06:36:02.855875 5020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:36:02.859228 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.856743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:33773 (local address 127.29.142.190:40745)
0806 06:36:02.856811 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.856813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.857026 (+ 213us) server_negotiation.cc:407] Connection header received
0806 06:36:02.857521 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.857522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.857528 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.857597 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:02.858994 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:02.858995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:02.858996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:02.859000 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:02.859015 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:02.859018 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:02.859072 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.859149 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.859171 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:36:02.859431 5198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.856687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:40745 (local address 127.29.142.131:33773)
0806 06:36:02.856884 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:02.856928 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:02.857414 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:02.857681 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:02.857684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:02.857693 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:02.858828 (+ 1135us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:02.858829 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:02.858860 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:02.858892 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:02.859071 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:02.859075 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:02.859077 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:36:02.859137 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:36:02.859194 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260806 06:36:02.859982 5194 heartbeater.cc:347] Connected to a master server at 127.29.142.190:40745
I20260806 06:36:02.860077 5194 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:02.860195 5194 heartbeater.cc:511] Master 127.29.142.190:40745 requested a full tablet report, sending...
I20260806 06:36:02.860625 4549 ts_manager.cc:194] Registered new tserver with Master: be19316564fa46c98a8fbb34d897fbb7 (127.29.142.131:39811)
I20260806 06:36:02.865715 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 5020
I20260806 06:36:02.865854 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:36:02.866050 30266 external_mini_cluster.cc:1428] Setting key b0700bde020a99c7b37ba2979d6c8324
I20260806 06:36:02.867237 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:36:02.870070 4604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.868702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.29.142.190:40745)
0806 06:36:02.868819 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.868822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.868928 (+ 106us) server_negotiation.cc:407] Connection header received
0806 06:36:02.869026 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.869027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.869034 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.869089 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:02.869819 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:02.869820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:02.869821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:02.869826 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:02.869842 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:02.869847 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:02.869914 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.869973 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.869990 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 06:36:02.871737 4549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:36:02.873353 4549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:36:02.877787 5208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.876317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.29.142.130:35521)
0806 06:36:02.876624 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.876629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.876640 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:02.876669 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.876671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.876682 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.876970 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:02.877465 (+ 495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:02.877468 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:02.877468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:02.877471 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:02.877501 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:02.877511 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:02.877599 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.877671 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.877689 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260806 06:36:02.877799 5205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.875860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:35521 (local address 127.0.0.1:41176)
0806 06:36:02.876029 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:02.876042 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:02.876111 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:02.876788 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:02.876789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:02.876792 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:02.877198 (+ 406us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:02.877198 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:02.877217 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:02.877244 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:02.877592 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:02.877598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:02.877601 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:36:02.877662 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:36:02.877673 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":117,"threads_started":1}
I20260806 06:36:02.877970 5209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.876602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56808 (local address 127.29.142.129:46869)
0806 06:36:02.876912 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.876916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.876927 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:02.876954 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.876956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.876965 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.877038 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:02.877569 (+ 531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:02.877579 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:02.877579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:02.877584 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:02.877622 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:02.877645 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:02.877708 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.877850 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.877869 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":99,"threads_started":1}
I20260806 06:36:02.878010 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.876145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:46869 (local address 127.0.0.1:56808)
0806 06:36:02.876293 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:02.876310 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:02.876352 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:02.877057 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:02.877057 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:02.877061 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:02.877465 (+ 404us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:02.877465 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:02.877470 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:02.877478 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:02.877716 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:02.877716 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:02.877717 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:36:02.877764 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:36:02.877776 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260806 06:36:02.878181 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.875808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:39811 (local address 127.0.0.1:58540)
0806 06:36:02.875982 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:02.876029 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:02.876091 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:02.876396 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:02.876406 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:02.876413 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:02.877455 (+ 1042us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:02.877455 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:02.877463 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:02.877477 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:02.877931 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:02.877933 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:02.877934 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:36:02.877969 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 06:36:02.877984 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
I20260806 06:36:02.878898 4726 tablet_service.cc:1467] Processing CreateTablet for tablet 199c4e508c62448eaeee1402ec9c16d1 (DEFAULT_TABLE table=test-table [id=5b43f6d7fd094f4bb74a92aee215ded2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:02.878898 4918 tablet_service.cc:1467] Processing CreateTablet for tablet 199c4e508c62448eaeee1402ec9c16d1 (DEFAULT_TABLE table=test-table [id=5b43f6d7fd094f4bb74a92aee215ded2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:02.879344 4726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199c4e508c62448eaeee1402ec9c16d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:02.879351 4918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199c4e508c62448eaeee1402ec9c16d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:02.882432 5211 tablet_bootstrap.cc:492] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72: Bootstrap starting.
I20260806 06:36:02.882474 5206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.876058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58540 (local address 127.29.142.131:39811)
0806 06:36:02.876234 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.876239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.876249 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:36:02.876275 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.876284 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.876300 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.876393 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:02.877751 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:02.877759 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:02.877760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:02.877764 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:02.877806 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:02.877836 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:02.878054 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.878076 (+ 22us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.878091 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260806 06:36:02.883002 5212 tablet_bootstrap.cc:492] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d: Bootstrap starting.
I20260806 06:36:02.883320 5211 tablet_bootstrap.cc:654] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:02.883291 5104 tablet_service.cc:1467] Processing CreateTablet for tablet 199c4e508c62448eaeee1402ec9c16d1 (DEFAULT_TABLE table=test-table [id=5b43f6d7fd094f4bb74a92aee215ded2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:02.883669 5104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199c4e508c62448eaeee1402ec9c16d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:02.883680 5211 log.cc:824] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:02.883746 5212 tablet_bootstrap.cc:654] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:02.884086 5212 log.cc:824] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:02.884804 5211 tablet_bootstrap.cc:492] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72: No bootstrap required, opened a new log
I20260806 06:36:02.884888 5211 ts_tablet_manager.cc:1397] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:36:02.884991 5212 tablet_bootstrap.cc:492] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d: No bootstrap required, opened a new log
I20260806 06:36:02.885058 5212 ts_tablet_manager.cc:1397] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:36:02.886936 5216 tablet_bootstrap.cc:492] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7: Bootstrap starting.
I20260806 06:36:02.887560 5216 tablet_bootstrap.cc:654] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:02.887789 5211 raft_consensus.cc:348] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be19316564fa46c98a8fbb34d897fbb7" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 39811 } } peers { permanent_uuid: "44475be6be544371b5fe1f4dd735c74d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46869 } } peers { permanent_uuid: "3c7f410432c14b8f9a93023364dfaf72" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35521 } }
I20260806 06:36:02.887939 5216 log.cc:824] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:02.887986 5211 raft_consensus.cc:374] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:02.888026 5211 raft_consensus.cc:729] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7f410432c14b8f9a93023364dfaf72, State: Initialized, Role: FOLLOWER
I20260806 06:36:02.888178 5211 consensus_queue.cc:260] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be19316564fa46c98a8fbb34d897fbb7" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 39811 } } peers { permanent_uuid: "44475be6be544371b5fe1f4dd735c74d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46869 } } peers { permanent_uuid: "3c7f410432c14b8f9a93023364dfaf72" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35521 } }
I20260806 06:36:02.888214 5212 raft_consensus.cc:348] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be19316564fa46c98a8fbb34d897fbb7" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 39811 } } peers { permanent_uuid: "44475be6be544371b5fe1f4dd735c74d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46869 } } peers { permanent_uuid: "3c7f410432c14b8f9a93023364dfaf72" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35521 } }
I20260806 06:36:02.888361 5212 raft_consensus.cc:374] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:02.888414 5212 raft_consensus.cc:729] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44475be6be544371b5fe1f4dd735c74d, State: Initialized, Role: FOLLOWER
I20260806 06:36:02.888515 5211 ts_tablet_manager.cc:1428] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 06:36:02.888547 5017 heartbeater.cc:503] Master 127.29.142.190:40745 was elected leader, sending a full tablet report...
I20260806 06:36:02.888558 5212 consensus_queue.cc:260] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be19316564fa46c98a8fbb34d897fbb7" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 39811 } } peers { permanent_uuid: "44475be6be544371b5fe1f4dd735c74d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46869 } } peers { permanent_uuid: "3c7f410432c14b8f9a93023364dfaf72" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35521 } }
I20260806 06:36:02.888842 5212 ts_tablet_manager.cc:1428] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 06:36:02.888878 4818 heartbeater.cc:503] Master 127.29.142.190:40745 was elected leader, sending a full tablet report...
I20260806 06:36:02.889876 5216 tablet_bootstrap.cc:492] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7: No bootstrap required, opened a new log
I20260806 06:36:02.889956 5216 ts_tablet_manager.cc:1397] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:36:02.891961 5216 raft_consensus.cc:348] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be19316564fa46c98a8fbb34d897fbb7" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 39811 } } peers { permanent_uuid: "44475be6be544371b5fe1f4dd735c74d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46869 } } peers { permanent_uuid: "3c7f410432c14b8f9a93023364dfaf72" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35521 } }
I20260806 06:36:02.892198 5216 raft_consensus.cc:374] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:02.892257 5216 raft_consensus.cc:729] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be19316564fa46c98a8fbb34d897fbb7, State: Initialized, Role: FOLLOWER
I20260806 06:36:02.892383 5216 consensus_queue.cc:260] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be19316564fa46c98a8fbb34d897fbb7" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 39811 } } peers { permanent_uuid: "44475be6be544371b5fe1f4dd735c74d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46869 } } peers { permanent_uuid: "3c7f410432c14b8f9a93023364dfaf72" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35521 } }
I20260806 06:36:02.892686 5216 ts_tablet_manager.cc:1428] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:36:02.892725 5194 heartbeater.cc:503] Master 127.29.142.190:40745 was elected leader, sending a full tablet report...
I20260806 06:36:02.906844 5223 raft_consensus.cc:482] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:36:02.907001 5223 raft_consensus.cc:504] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be19316564fa46c98a8fbb34d897fbb7" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 39811 } } peers { permanent_uuid: "44475be6be544371b5fe1f4dd735c74d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46869 } } peers { permanent_uuid: "3c7f410432c14b8f9a93023364dfaf72" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35521 } }
I20260806 06:36:02.907459 5223 leader_election.cc:288] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44475be6be544371b5fe1f4dd735c74d (127.29.142.129:46869), 3c7f410432c14b8f9a93023364dfaf72 (127.29.142.130:35521)
I20260806 06:36:02.908878 5198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.907699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:35521 (local address 127.29.142.131:38549)
0806 06:36:02.907752 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:02.907766 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:02.907807 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:02.908020 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:02.908021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:02.908023 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:02.908502 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:02.908503 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:02.908510 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:02.908523 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:02.908734 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:02.908735 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:02.908736 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:36:02.908792 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:36:02.908807 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:36:02.908893 5208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.907793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:38549 (local address 127.29.142.130:35521)
0806 06:36:02.907851 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.907853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.907865 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:36:02.907902 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.907903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.907908 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.908005 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:02.908623 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:02.908624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:02.908625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:02.908630 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:02.908652 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:02.908657 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:02.908722 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.908808 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.908822 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:36:02.909318 4945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199c4e508c62448eaeee1402ec9c16d1" candidate_uuid: "be19316564fa46c98a8fbb34d897fbb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c7f410432c14b8f9a93023364dfaf72" is_pre_election: true
I20260806 06:36:02.909488 5209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.907810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:41517 (local address 127.29.142.129:46869)
0806 06:36:02.907856 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.907858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.908106 (+ 248us) server_negotiation.cc:407] Connection header received
0806 06:36:02.908142 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.908143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.908150 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.908214 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:02.909254 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:02.909256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:02.909256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:02.909263 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:02.909290 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:02.909296 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:02.909363 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.909411 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.909425 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:36:02.909511 4945 raft_consensus.cc:2454] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be19316564fa46c98a8fbb34d897fbb7 in term 0.
I20260806 06:36:02.909534 5227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.907706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:46869 (local address 127.29.142.131:41517)
0806 06:36:02.908016 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:02.908027 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:02.908090 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:02.908490 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:02.908491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:02.908493 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:02.909118 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:02.909118 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:02.909125 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:02.909140 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:02.909367 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:02.909367 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:02.909368 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:36:02.909410 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:36:02.909421 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":51,"thread_start_us":168,"threads_started":1}
I20260806 06:36:02.909857 5038 leader_election.cc:302] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c7f410432c14b8f9a93023364dfaf72, be19316564fa46c98a8fbb34d897fbb7; no voters:
I20260806 06:36:02.909897 4756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199c4e508c62448eaeee1402ec9c16d1" candidate_uuid: "be19316564fa46c98a8fbb34d897fbb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44475be6be544371b5fe1f4dd735c74d" is_pre_election: true
I20260806 06:36:02.910032 4756 raft_consensus.cc:2454] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be19316564fa46c98a8fbb34d897fbb7 in term 0.
I20260806 06:36:02.910027 5223 raft_consensus.cc:2781] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:36:02.910076 5223 raft_consensus.cc:482] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:36:02.910109 5223 raft_consensus.cc:3037] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:02.911026 5223 raft_consensus.cc:504] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be19316564fa46c98a8fbb34d897fbb7" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 39811 } } peers { permanent_uuid: "44475be6be544371b5fe1f4dd735c74d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46869 } } peers { permanent_uuid: "3c7f410432c14b8f9a93023364dfaf72" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35521 } }
I20260806 06:36:02.911270 5223 leader_election.cc:288] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [CANDIDATE]: Term 1 election: Requested vote from peers 44475be6be544371b5fe1f4dd735c74d (127.29.142.129:46869), 3c7f410432c14b8f9a93023364dfaf72 (127.29.142.130:35521)
I20260806 06:36:02.911407 4756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199c4e508c62448eaeee1402ec9c16d1" candidate_uuid: "be19316564fa46c98a8fbb34d897fbb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44475be6be544371b5fe1f4dd735c74d"
I20260806 06:36:02.911494 4945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199c4e508c62448eaeee1402ec9c16d1" candidate_uuid: "be19316564fa46c98a8fbb34d897fbb7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c7f410432c14b8f9a93023364dfaf72"
I20260806 06:36:02.911523 4756 raft_consensus.cc:3037] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:02.911567 4945 raft_consensus.cc:3037] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:02.912559 4756 raft_consensus.cc:2454] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be19316564fa46c98a8fbb34d897fbb7 in term 1.
I20260806 06:36:02.912559 4945 raft_consensus.cc:2454] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be19316564fa46c98a8fbb34d897fbb7 in term 1.
I20260806 06:36:02.912750 5040 leader_election.cc:302] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44475be6be544371b5fe1f4dd735c74d, be19316564fa46c98a8fbb34d897fbb7; no voters:
I20260806 06:36:02.912878 5223 raft_consensus.cc:2781] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:02.913074 5223 raft_consensus.cc:686] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 1 LEADER]: Becoming Leader. State: Replica: be19316564fa46c98a8fbb34d897fbb7, State: Running, Role: LEADER
I20260806 06:36:02.913192 5223 consensus_queue.cc:237] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be19316564fa46c98a8fbb34d897fbb7" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 39811 } } peers { permanent_uuid: "44475be6be544371b5fe1f4dd735c74d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46869 } } peers { permanent_uuid: "3c7f410432c14b8f9a93023364dfaf72" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35521 } }
I20260806 06:36:02.914156 4549 catalog_manager.cc:5495] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 reported cstate change: term changed from 0 to 1, leader changed from <none> to be19316564fa46c98a8fbb34d897fbb7 (127.29.142.131). New cstate: current_term: 1 leader_uuid: "be19316564fa46c98a8fbb34d897fbb7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be19316564fa46c98a8fbb34d897fbb7" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 39811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44475be6be544371b5fe1f4dd735c74d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c7f410432c14b8f9a93023364dfaf72" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35521 } health_report { overall_health: UNKNOWN } } }
I20260806 06:36:02.926036 5206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.924872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58554 (local address 127.29.142.131:39811)
0806 06:36:02.924941 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:02.924944 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:02.924952 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:36:02.925001 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:02.925001 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:02.925015 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:02.925094 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:02.925773 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:02.925774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:02.925775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:02.925787 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:02.925806 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:02.925810 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:02.925881 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:02.925938 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 06:36:02.925953 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:36:02.928202 4756 raft_consensus.cc:1261] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d [term 1 FOLLOWER]: Refusing update from remote peer be19316564fa46c98a8fbb34d897fbb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:02.928202 4945 raft_consensus.cc:1261] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72 [term 1 FOLLOWER]: Refusing update from remote peer be19316564fa46c98a8fbb34d897fbb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:02.928610 5228 consensus_queue.cc:1035] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44475be6be544371b5fe1f4dd735c74d" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:02.928768 5223 consensus_queue.cc:1035] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c7f410432c14b8f9a93023364dfaf72" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:02.931488 5233 mvcc.cc:205] Tried to move back new op lower bound from 7315448475350364160 to 7315448475293257728. Current Snapshot: MvccSnapshot[applied={T|T < 7315448475350364160}]
I20260806 06:36:02.934029 4549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:02.934134 4549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:02.934969 4549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 662ce1de341f40e7a01854587e7c38b3: Sending DeleteTablet for 3 replicas of tablet 199c4e508c62448eaeee1402ec9c16d1
I20260806 06:36:02.935287 5104 tablet_service.cc:1514] Processing DeleteTablet for tablet 199c4e508c62448eaeee1402ec9c16d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:02 UTC) from {username='slave'} at 127.0.0.1:58540
I20260806 06:36:02.935346 4918 tablet_service.cc:1514] Processing DeleteTablet for tablet 199c4e508c62448eaeee1402ec9c16d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:02 UTC) from {username='slave'} at 127.0.0.1:41176
I20260806 06:36:02.935350 4726 tablet_service.cc:1514] Processing DeleteTablet for tablet 199c4e508c62448eaeee1402ec9c16d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:02 UTC) from {username='slave'} at 127.0.0.1:56808
I20260806 06:36:02.935581 5250 tablet_replica.cc:330] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d: stopping tablet replica
I20260806 06:36:02.935699 5250 raft_consensus.cc:2229] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:02.935819 5250 raft_consensus.cc:2258] T 199c4e508c62448eaeee1402ec9c16d1 P 44475be6be544371b5fe1f4dd735c74d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:02.935937 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 4636
I20260806 06:36:02.936002 5247 tablet_replica.cc:330] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7: stopping tablet replica
I20260806 06:36:02.936632 5249 tablet_replica.cc:330] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72: stopping tablet replica
I20260806 06:36:02.936790 5249 raft_consensus.cc:2229] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:02.936944 5249 raft_consensus.cc:2258] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:02.937105 5247 raft_consensus.cc:2229] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:36:02.937282 5247 raft_consensus.cc:2258] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:02.938884 5247 ts_tablet_manager.cc:1905] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:02.938892 5249 ts_tablet_manager.cc:1905] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:02.940838 5249 ts_tablet_manager.cc:1918] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:02.941015 5249 log.cc:1196] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/199c4e508c62448eaeee1402ec9c16d1
I20260806 06:36:02.941040 5247 ts_tablet_manager.cc:1918] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:02.941123 5247 log.cc:1196] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/199c4e508c62448eaeee1402ec9c16d1
I20260806 06:36:02.941450 5247 ts_tablet_manager.cc:1939] T 199c4e508c62448eaeee1402ec9c16d1 P be19316564fa46c98a8fbb34d897fbb7: Deleting consensus metadata
I20260806 06:36:02.941562 5249 ts_tablet_manager.cc:1939] T 199c4e508c62448eaeee1402ec9c16d1 P 3c7f410432c14b8f9a93023364dfaf72: Deleting consensus metadata
I20260806 06:36:02.941983 4528 catalog_manager.cc:4841] TS be19316564fa46c98a8fbb34d897fbb7 (127.29.142.131:39811): tablet 199c4e508c62448eaeee1402ec9c16d1 (table test-table [id=5b43f6d7fd094f4bb74a92aee215ded2]) successfully deleted
W20260806 06:36:02.945667 4528 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:46869 (error 108)
I20260806 06:36:02.945919 4524 catalog_manager.cc:4841] TS 3c7f410432c14b8f9a93023364dfaf72 (127.29.142.130:35521): tablet 199c4e508c62448eaeee1402ec9c16d1 (table test-table [id=5b43f6d7fd094f4bb74a92aee215ded2]) successfully deleted
I20260806 06:36:02.946358 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:02.946187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:46869 (local address 127.0.0.1:56818)
0806 06:36:02.946260 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:02.946321 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 06:36:02.946376 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 4822
W20260806 06:36:02.946491 4528 catalog_manager.cc:4584] TS 44475be6be544371b5fe1f4dd735c74d (127.29.142.129:46869): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 199c4e508c62448eaeee1402ec9c16d1: Network error: Client connection negotiation failed: client connection to 127.29.142.129:46869: connect: Connection refused (error 111)
I20260806 06:36:02.963661 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 5020
I20260806 06:36:02.977612 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 4499
2026-08-06T06:36:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (653 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 06:36:03.005158 30266 mini_oidc.cc:152] Starting JWKS server
I20260806 06:36:03.006474 30266 webserver.cc:460] Webserver started at https://127.0.0.1:39919/ using document root <none> and password file <none>
I20260806 06:36:03.006618 30266 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:36:03.006883 30266 webserver.cc:460] Webserver started at http://127.0.0.1:41883/ using document root <none> and password file <none>
2026-08-06T06:36:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:03Z Disabled control of system clock
I20260806 06:36:03.016705 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:39713
--webserver_interface=127.29.142.190
--webserver_port=0
--builtin_ntp_servers=127.29.142.148:37047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:39713
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39919/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/testchainca.cert with env {}
W20260806 06:36:03.103358 5258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:03.103611 5258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:03.103637 5258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:03.104931 5258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:03.104990 5258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:03.105005 5258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:03.105017 5258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:03.105031 5258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:36:03.105043 5258 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:36:03.105055 5258 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39919/jwks.json
I20260806 06:36:03.106905 5258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:37047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:39713
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:39713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39919/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:03.107209 5258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:03.107466 5258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:03.112896 5266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:03.112995 5258 server_base.cc:1034] running on GCE node
W20260806 06:36:03.112999 5267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:03.112896 5271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:03.113452 5258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:03.113750 5258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:03.114930 5258 hybrid_clock.cc:648] HybridClock initialized: now 1785998163114906 us; error 44 us; skew 500 ppm
I20260806 06:36:03.116374 5258 webserver.cc:460] Webserver started at http://127.29.142.190:43795/ using document root <none> and password file <none>
I20260806 06:36:03.116663 5258 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:03.116717 5258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:03.116887 5258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:03.118022 5258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "5c39a3311a364247b9b9570232046943"
format_stamp: "Formatted at 2026-08-06 06:36:03 on dist-test-slave-3kk6"
server_key: "122b866f954ef24de7f2a19a18bbf133"
server_key_iv: "d4bdf80c426d1695c519bea84cfa9f40"
server_key_version: "encryptionkey@0"
I20260806 06:36:03.118423 5258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "5c39a3311a364247b9b9570232046943"
format_stamp: "Formatted at 2026-08-06 06:36:03 on dist-test-slave-3kk6"
server_key: "122b866f954ef24de7f2a19a18bbf133"
server_key_iv: "d4bdf80c426d1695c519bea84cfa9f40"
server_key_version: "encryptionkey@0"
I20260806 06:36:03.119925 5258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:03.120741 5281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:03.120946 5258 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:03.121054 5258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "5c39a3311a364247b9b9570232046943"
format_stamp: "Formatted at 2026-08-06 06:36:03 on dist-test-slave-3kk6"
server_key: "122b866f954ef24de7f2a19a18bbf133"
server_key_iv: "d4bdf80c426d1695c519bea84cfa9f40"
server_key_version: "encryptionkey@0"
I20260806 06:36:03.121145 5258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:03.143075 5258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:03.143486 5258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:03.143671 5258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:03.148101 5258 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:39713
I20260806 06:36:03.148141 5358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:39713 every 8 connection(s)
I20260806 06:36:03.148545 5258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:03.149705 5359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:03.152272 5359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943: Bootstrap starting.
I20260806 06:36:03.152566 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 5258
I20260806 06:36:03.152704 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:03.152895 30266 external_mini_cluster.cc:1428] Setting key 3801ac45bf64d867cdd88bb03291db19
I20260806 06:36:03.153127 5359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:03.153483 5359 log.cc:824] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:03.154326 5359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943: No bootstrap required, opened a new log
I20260806 06:36:03.156889 5359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c39a3311a364247b9b9570232046943" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39713 } }
I20260806 06:36:03.157050 5359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:03.157075 5359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c39a3311a364247b9b9570232046943, State: Initialized, Role: FOLLOWER
I20260806 06:36:03.157219 5359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c39a3311a364247b9b9570232046943" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39713 } }
I20260806 06:36:03.157308 5359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:03.157346 5359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:03.157399 5359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:03.158036 5359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c39a3311a364247b9b9570232046943" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39713 } }
I20260806 06:36:03.158171 5359 leader_election.cc:302] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c39a3311a364247b9b9570232046943; no voters:
I20260806 06:36:03.158424 5359 leader_election.cc:288] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:03.158458 5365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:03.158784 5359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:03.158911 5365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [term 1 LEADER]: Becoming Leader. State: Replica: 5c39a3311a364247b9b9570232046943, State: Running, Role: LEADER
I20260806 06:36:03.159096 5365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c39a3311a364247b9b9570232046943" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39713 } }
I20260806 06:36:03.159485 5372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c39a3311a364247b9b9570232046943. Latest consensus state: current_term: 1 leader_uuid: "5c39a3311a364247b9b9570232046943" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c39a3311a364247b9b9570232046943" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39713 } } }
I20260806 06:36:03.159483 5371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c39a3311a364247b9b9570232046943" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c39a3311a364247b9b9570232046943" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39713 } } }
I20260806 06:36:03.159687 5372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:03.159698 5371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:03.159993 5362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:03.153535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33430 (local address 127.29.142.190:39713)
0806 06:36:03.153780 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:03.153788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:03.153816 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:36:03.154418 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:03.154429 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:03.154518 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:03.154705 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:03.154959 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.156385 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.157275 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.157830 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.157984 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:03.158860 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:03.158865 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:03.158866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:03.159099 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:03.159146 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:03.159163 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:03.159268 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:03.159631 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 06:36:03.159745 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":96,"threads_started":1}
I20260806 06:36:03.161144 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 06:36:03.161145 5389 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:36:03.161221 5389 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:36:03.161278 5381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:03.161446 5381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:03.163110 5381 catalog_manager.cc:1347] Generated new cluster ID: 1d5ea6f9f6a94a4bafdd17819cd27326
I20260806 06:36:03.163170 5381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:36:03.171737 5381 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:03.172475 5381 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:03.181356 5381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c39a3311a364247b9b9570232046943: Generated new TSK 0
I20260806 06:36:03.181622 5381 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 06:36:03.207006 5404 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 06:36:03.207206 5404 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 06:36:03.207481 5362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:03.191417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33432 (local address 127.29.142.190:39713)
0806 06:36:03.191473 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:03.191475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:03.191609 (+ 134us) server_negotiation.cc:407] Connection header received
0806 06:36:03.191750 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:03.191750 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:03.191761 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:03.191828 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:36:03.192001 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.192529 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.193731 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.193946 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.194009 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:03.194258 (+ 249us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:36:03.207119 (+ 12861us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39919/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 06:36:03.207384 (+ 265us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33432: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39919/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 06:36:03.207577 5362 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33432: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39919/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 06:36:03.208016 5403 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:03.191304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39713 (local address 127.0.0.1:33432)
0806 06:36:03.191518 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:03.191532 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:03.191623 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:03.191853 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:03.191854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:03.191857 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:36:03.191932 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:03.191935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.192624 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.192625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:03.193628 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:03.193631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.194045 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.194046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:03.194109 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:03.194140 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:36:03.207420 (+ 13280us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39919/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 06:36:03.207579 (+ 159us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.142.190:39713: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39919/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":116,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.29.142.190:39713: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39919/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 06:36:03.208498 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 5258
2026-08-06T06:36:03Z chronyd exiting
I20260806 06:36:03.252480 30266 test_util.cc:175] -----------------------------------------------
I20260806 06:36:03.252570 30266 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998137760399-30266-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (248 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 06:36:03.253309 30266 mini_oidc.cc:152] Starting JWKS server
I20260806 06:36:03.254016 30266 webserver.cc:460] Webserver started at https://127.0.0.1:44935/ using document root <none> and password file <none>
I20260806 06:36:03.254060 30266 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:36:03.254174 30266 webserver.cc:460] Webserver started at http://127.0.0.1:33151/ using document root <none> and password file <none>
2026-08-06T06:36:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:03Z Disabled control of system clock
I20260806 06:36:03.264299 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:45037
--webserver_interface=127.29.142.190
--webserver_port=0
--builtin_ntp_servers=127.29.142.148:40921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:45037
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44935/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/testchainca.cert with env {}
W20260806 06:36:03.397514 5410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:03.397805 5410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:03.397851 5410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:03.399245 5410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:03.399322 5410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:03.399343 5410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:03.399364 5410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:03.399398 5410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:36:03.399430 5410 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:36:03.399451 5410 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44935/jwks.json
I20260806 06:36:03.402139 5410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:45037
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:45037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44935/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:03.402459 5410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:03.402796 5410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:03.408144 5419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:03.408156 5422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:03.408155 5418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:03.408787 5410 server_base.cc:1034] running on GCE node
I20260806 06:36:03.409098 5410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:03.409432 5410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:03.410614 5410 hybrid_clock.cc:648] HybridClock initialized: now 1785998163410595 us; error 51 us; skew 500 ppm
I20260806 06:36:03.412063 5410 webserver.cc:460] Webserver started at http://127.29.142.190:33613/ using document root <none> and password file <none>
I20260806 06:36:03.412364 5410 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:03.412443 5410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:03.412609 5410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:03.413757 5410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "31e9c82e782943179cb41a38cb0afc4f"
format_stamp: "Formatted at 2026-08-06 06:36:03 on dist-test-slave-3kk6"
server_key: "658afbf754f3fc953c3f54a6775a5fd3"
server_key_iv: "4adb23e4598260d47805e65d57a18e57"
server_key_version: "encryptionkey@0"
I20260806 06:36:03.414168 5410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "31e9c82e782943179cb41a38cb0afc4f"
format_stamp: "Formatted at 2026-08-06 06:36:03 on dist-test-slave-3kk6"
server_key: "658afbf754f3fc953c3f54a6775a5fd3"
server_key_iv: "4adb23e4598260d47805e65d57a18e57"
server_key_version: "encryptionkey@0"
I20260806 06:36:03.415659 5410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:03.416437 5431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:03.416615 5410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:03.416730 5410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "31e9c82e782943179cb41a38cb0afc4f"
format_stamp: "Formatted at 2026-08-06 06:36:03 on dist-test-slave-3kk6"
server_key: "658afbf754f3fc953c3f54a6775a5fd3"
server_key_iv: "4adb23e4598260d47805e65d57a18e57"
server_key_version: "encryptionkey@0"
I20260806 06:36:03.416815 5410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:03.434233 5410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:03.434614 5410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:03.434805 5410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:03.439225 5410 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:45037
I20260806 06:36:03.439307 5508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:45037 every 8 connection(s)
I20260806 06:36:03.439711 5410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:03.440510 5509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:03.440521 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 5410
I20260806 06:36:03.440646 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:03.440843 30266 external_mini_cluster.cc:1428] Setting key 4fa0d1dd7ed9d6bf16157e8c5d7075f9
I20260806 06:36:03.444989 5509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f: Bootstrap starting.
I20260806 06:36:03.445760 5509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:03.446103 5509 log.cc:824] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:03.447188 5509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f: No bootstrap required, opened a new log
I20260806 06:36:03.447448 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:03.441521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54304 (local address 127.29.142.190:45037)
0806 06:36:03.441749 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:03.441757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:03.441789 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:36:03.442515 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:03.442526 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:03.442606 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:03.442805 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:03.443045 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.443962 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.445128 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.445571 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.445689 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:03.446542 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:03.446547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:03.446548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:03.446870 (+ 322us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:03.446906 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:03.446919 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:03.446978 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:03.447100 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 06:36:03.447212 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260806 06:36:03.448539 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:36:03.449599 5509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e9c82e782943179cb41a38cb0afc4f" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45037 } }
I20260806 06:36:03.449810 5509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:03.449863 5509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e9c82e782943179cb41a38cb0afc4f, State: Initialized, Role: FOLLOWER
I20260806 06:36:03.449996 5509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e9c82e782943179cb41a38cb0afc4f" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45037 } }
I20260806 06:36:03.450095 5509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:03.450142 5509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:03.450196 5509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:03.488514 5509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e9c82e782943179cb41a38cb0afc4f" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45037 } }
I20260806 06:36:03.488709 5509 leader_election.cc:302] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31e9c82e782943179cb41a38cb0afc4f; no voters:
I20260806 06:36:03.489004 5509 leader_election.cc:288] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:03.489107 5515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:03.489346 5515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [term 1 LEADER]: Becoming Leader. State: Replica: 31e9c82e782943179cb41a38cb0afc4f, State: Running, Role: LEADER
I20260806 06:36:03.489382 5509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:03.489476 5515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e9c82e782943179cb41a38cb0afc4f" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45037 } }
I20260806 06:36:03.490005 5516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31e9c82e782943179cb41a38cb0afc4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e9c82e782943179cb41a38cb0afc4f" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45037 } } }
I20260806 06:36:03.490120 5516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:03.490404 5518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31e9c82e782943179cb41a38cb0afc4f. Latest consensus state: current_term: 1 leader_uuid: "31e9c82e782943179cb41a38cb0afc4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e9c82e782943179cb41a38cb0afc4f" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 45037 } } }
I20260806 06:36:03.490522 5518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:03.490878 5528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:03.491494 5528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:03.493503 5528 catalog_manager.cc:1347] Generated new cluster ID: 414ea92941a746c0a18d3ba36a094127
I20260806 06:36:03.493584 5528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:36:03.505246 5528 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:03.505833 5528 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:03.509101 5528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31e9c82e782943179cb41a38cb0afc4f: Generated new TSK 0
I20260806 06:36:03.509317 5528 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 06:36:03.540495 5554 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 06:36:03.540599 5554 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 06:36:03.540906 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:03.517208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54310 (local address 127.29.142.190:45037)
0806 06:36:03.517283 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:03.517286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:03.517388 (+ 102us) server_negotiation.cc:407] Connection header received
0806 06:36:03.517477 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:03.517478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:03.517490 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:03.517561 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:36:03.517776 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.518384 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.519609 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.519871 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.519949 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:03.520179 (+ 230us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:36:03.540652 (+ 20473us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44935/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 06:36:03.540847 (+ 195us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54310: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44935/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 06:36:03.540992 5513 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54310: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44935/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 06:36:03.541057 5552 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:03.517102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:45037 (local address 127.0.0.1:54310)
0806 06:36:03.517297 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:03.517310 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:03.517393 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:03.517582 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:03.517583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:03.517585 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:36:03.517663 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:03.517665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.518483 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.518484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:03.519498 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:03.519501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.519963 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.519964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:03.520026 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:03.520033 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:36:03.540793 (+ 20760us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44935/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 06:36:03.540955 (+ 162us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.142.190:45037: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44935/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":118,"threads_started":1}
I20260806 06:36:03.541455 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 5410
2026-08-06T06:36:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (331 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 06:36:03.584657 30266 mini_oidc.cc:152] Starting JWKS server
I20260806 06:36:03.585436 30266 webserver.cc:460] Webserver started at https://127.0.0.1:42625/ using document root <none> and password file <none>
I20260806 06:36:03.585513 30266 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:36:03.585731 30266 webserver.cc:460] Webserver started at http://127.0.0.1:38977/ using document root <none> and password file <none>
2026-08-06T06:36:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:03Z Disabled control of system clock
I20260806 06:36:03.596192 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:46271
--webserver_interface=127.29.142.190
--webserver_port=0
--builtin_ntp_servers=127.29.142.148:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:46271
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42625/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 06:36:03.685775 5564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:03.686007 5564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:03.686030 5564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:03.687307 5564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:03.687358 5564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:03.687371 5564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:03.687384 5564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:03.687397 5564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:36:03.687409 5564 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:36:03.687458 5564 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42625/jwks.json
I20260806 06:36:03.689059 5564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:46271
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:46271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42625/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:03.689333 5564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:03.689643 5564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:03.694993 5576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:03.694993 5578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:03.696369 5571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:03.697781 5564 server_base.cc:1034] running on GCE node
I20260806 06:36:03.698099 5564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:03.698475 5564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:03.699687 5564 hybrid_clock.cc:648] HybridClock initialized: now 1785998163699686 us; error 59 us; skew 500 ppm
I20260806 06:36:03.701259 5564 webserver.cc:460] Webserver started at http://127.29.142.190:39593/ using document root <none> and password file <none>
I20260806 06:36:03.701629 5564 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:03.701721 5564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:03.701879 5564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:03.703186 5564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "d5ac8ff8716e426aa1930f665fdd1cde"
format_stamp: "Formatted at 2026-08-06 06:36:03 on dist-test-slave-3kk6"
server_key: "bde9bab5c3f3ecac27ff8518b689e47b"
server_key_iv: "f8bde30ce88d709270d315b82c36db9c"
server_key_version: "encryptionkey@0"
I20260806 06:36:03.703606 5564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "d5ac8ff8716e426aa1930f665fdd1cde"
format_stamp: "Formatted at 2026-08-06 06:36:03 on dist-test-slave-3kk6"
server_key: "bde9bab5c3f3ecac27ff8518b689e47b"
server_key_iv: "f8bde30ce88d709270d315b82c36db9c"
server_key_version: "encryptionkey@0"
I20260806 06:36:03.705116 5564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:36:03.706051 5584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:03.706271 5564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:03.706377 5564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "d5ac8ff8716e426aa1930f665fdd1cde"
format_stamp: "Formatted at 2026-08-06 06:36:03 on dist-test-slave-3kk6"
server_key: "bde9bab5c3f3ecac27ff8518b689e47b"
server_key_iv: "f8bde30ce88d709270d315b82c36db9c"
server_key_version: "encryptionkey@0"
I20260806 06:36:03.706467 5564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:03.731667 5564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:03.732112 5564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:03.732308 5564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:03.736858 5564 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:46271
I20260806 06:36:03.736907 5661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:46271 every 8 connection(s)
I20260806 06:36:03.737314 5564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:03.738081 5662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:03.741096 5662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde: Bootstrap starting.
I20260806 06:36:03.741765 5662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:03.742071 5662 log.cc:824] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:03.742313 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 5564
I20260806 06:36:03.742427 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:03.742594 30266 external_mini_cluster.cc:1428] Setting key 97c3909fe9d9c6860dd5af329ca3ce51
I20260806 06:36:03.742967 5662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde: No bootstrap required, opened a new log
I20260806 06:36:03.744975 5662 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ac8ff8716e426aa1930f665fdd1cde" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46271 } }
I20260806 06:36:03.745148 5662 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:03.745199 5662 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5ac8ff8716e426aa1930f665fdd1cde, State: Initialized, Role: FOLLOWER
I20260806 06:36:03.745386 5662 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ac8ff8716e426aa1930f665fdd1cde" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46271 } }
I20260806 06:36:03.745500 5662 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:03.745548 5662 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:03.745600 5662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:03.749081 5668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:03.743319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51766 (local address 127.29.142.190:46271)
0806 06:36:03.743584 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:03.743595 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:03.743666 (+ 71us) server_negotiation.cc:407] Connection header received
0806 06:36:03.744297 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:03.744304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:03.744368 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:03.744534 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:03.744762 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.746187 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.747070 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.747539 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.747674 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:03.748427 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:03.748436 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:03.748436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:03.748615 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:03.748647 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:03.748657 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:03.748731 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:03.748847 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 06:36:03.748930 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":88,"threads_started":1}
I20260806 06:36:03.750016 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:36:03.789590 5662 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ac8ff8716e426aa1930f665fdd1cde" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46271 } }
I20260806 06:36:03.789825 5662 leader_election.cc:302] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5ac8ff8716e426aa1930f665fdd1cde; no voters:
I20260806 06:36:03.790086 5662 leader_election.cc:288] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:03.790195 5669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:03.790390 5669 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [term 1 LEADER]: Becoming Leader. State: Replica: d5ac8ff8716e426aa1930f665fdd1cde, State: Running, Role: LEADER
I20260806 06:36:03.790395 5662 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:03.790514 5669 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ac8ff8716e426aa1930f665fdd1cde" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46271 } }
I20260806 06:36:03.790994 5669 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5ac8ff8716e426aa1930f665fdd1cde. Latest consensus state: current_term: 1 leader_uuid: "d5ac8ff8716e426aa1930f665fdd1cde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ac8ff8716e426aa1930f665fdd1cde" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46271 } } }
I20260806 06:36:03.791103 5669 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:03.791003 5670 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5ac8ff8716e426aa1930f665fdd1cde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ac8ff8716e426aa1930f665fdd1cde" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46271 } } }
I20260806 06:36:03.791217 5670 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde [sys.catalog]: This master's current role is: LEADER
W20260806 06:36:03.792935 5693 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:36:03.793005 5693 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:36:03.793053 5683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:03.793220 5683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:03.795914 5683 catalog_manager.cc:1347] Generated new cluster ID: 474a653b2100459da6d0444ecb12643b
I20260806 06:36:03.795971 5683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:36:03.811115 5683 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:03.811828 5683 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:03.815059 5683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5ac8ff8716e426aa1930f665fdd1cde: Generated new TSK 0
I20260806 06:36:03.815255 5683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:36:03.836894 5668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:03.832707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51768 (local address 127.29.142.190:46271)
0806 06:36:03.832774 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:03.832777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:03.832882 (+ 105us) server_negotiation.cc:407] Connection header received
0806 06:36:03.832945 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:03.832946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:03.832961 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:03.833053 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:36:03.833184 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.834010 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.834895 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.835234 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.835314 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:03.835515 (+ 201us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:36:03.836658 (+ 1143us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 06:36:03.836810 (+ 152us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51768: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 06:36:03.836966 5668 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51768: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 06:36:03.838853 5703 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:03.832609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:46271 (local address 127.0.0.1:51768)
0806 06:36:03.832798 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:03.832811 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:03.832881 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:03.833058 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:03.833058 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:03.833060 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:36:03.833120 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:03.833122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.834101 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.834101 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:03.834809 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:03.834812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:03.835324 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:03.835325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:03.835399 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:03.835405 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:36:03.836791 (+ 1386us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 06:36:03.838735 (+ 1944us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.142.190:46271: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42625/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":1715,"thread_start_us":115,"threads_started":1}
I20260806 06:36:03.839035 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 5564
E20260806 06:36:03.854312 5704 webserver.cc:533] Webserver: sslize failed:
E20260806 06:36:03.854406 5704 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T06:36:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (301 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T06:36:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:03Z Disabled control of system clock
I20260806 06:36:03.897683 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:40747
--webserver_interface=127.29.142.190
--webserver_port=0
--builtin_ntp_servers=127.29.142.148:46363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:40747
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/testchainca.cert with env {}
W20260806 06:36:03.987497 5711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:03.987742 5711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:03.987766 5711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:03.989073 5711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:03.989156 5711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:03.989200 5711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:03.989238 5711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:03.989264 5711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:36:03.989303 5711 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:36:03.989344 5711 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 06:36:03.991168 5711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:46363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:40747
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:40747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:03.991467 5711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:03.991731 5711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:03.997499 5720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:03.997476 5718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:03.997681 5711 server_base.cc:1034] running on GCE node
W20260806 06:36:03.997532 5723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:03.998135 5711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:03.998521 5711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:04.001792 5711 hybrid_clock.cc:648] HybridClock initialized: now 1785998164001743 us; error 48 us; skew 500 ppm
I20260806 06:36:04.003480 5711 webserver.cc:460] Webserver started at http://127.29.142.190:43167/ using document root <none> and password file <none>
I20260806 06:36:04.003726 5711 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:04.003772 5711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:04.003881 5711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:04.005105 5711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "673943a7cd614424a522db1b12d7b4f0"
format_stamp: "Formatted at 2026-08-06 06:36:04 on dist-test-slave-3kk6"
server_key: "d292871cc4d9f31cedf78918b65f8848"
server_key_iv: "829de19c319160c6b4d44193c8d4c19f"
server_key_version: "encryptionkey@0"
I20260806 06:36:04.005446 5711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "673943a7cd614424a522db1b12d7b4f0"
format_stamp: "Formatted at 2026-08-06 06:36:04 on dist-test-slave-3kk6"
server_key: "d292871cc4d9f31cedf78918b65f8848"
server_key_iv: "829de19c319160c6b4d44193c8d4c19f"
server_key_version: "encryptionkey@0"
I20260806 06:36:04.006860 5711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260806 06:36:04.007704 5731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:04.007889 5711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:04.007990 5711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "673943a7cd614424a522db1b12d7b4f0"
format_stamp: "Formatted at 2026-08-06 06:36:04 on dist-test-slave-3kk6"
server_key: "d292871cc4d9f31cedf78918b65f8848"
server_key_iv: "829de19c319160c6b4d44193c8d4c19f"
server_key_version: "encryptionkey@0"
I20260806 06:36:04.008074 5711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:04.026885 5711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:04.027377 5711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:04.027609 5711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:04.032518 5711 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:40747
I20260806 06:36:04.032557 5803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:40747 every 8 connection(s)
I20260806 06:36:04.032975 5711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:04.034047 5804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:04.035184 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 5711
I20260806 06:36:04.035290 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:04.035504 30266 external_mini_cluster.cc:1428] Setting key f8b8ad36eef3d936c7dda3329c75a262
I20260806 06:36:04.037436 5804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0: Bootstrap starting.
I20260806 06:36:04.038220 5804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:04.038553 5804 log.cc:824] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:04.039394 5804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0: No bootstrap required, opened a new log
I20260806 06:36:04.041841 5804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "673943a7cd614424a522db1b12d7b4f0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40747 } }
I20260806 06:36:04.042047 5804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:04.042107 5804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 673943a7cd614424a522db1b12d7b4f0, State: Initialized, Role: FOLLOWER
I20260806 06:36:04.042238 5804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "673943a7cd614424a522db1b12d7b4f0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40747 } }
I20260806 06:36:04.042289 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:04.036124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39960 (local address 127.29.142.190:40747)
0806 06:36:04.036403 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:04.036413 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:04.036448 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:36:04.037130 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:04.037140 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:04.037227 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:04.037407 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:04.037648 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:04.039073 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:04.039927 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:04.040581 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:04.040727 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:04.041501 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:04.041507 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:04.041507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:04.041746 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:04.041791 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:04.041807 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:04.041888 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:04.041954 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 06:36:04.042041 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":128,"threads_started":1}
I20260806 06:36:04.042348 5804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:04.042385 5804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:04.042433 5804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:04.043411 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:36:04.093122 5804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "673943a7cd614424a522db1b12d7b4f0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40747 } }
I20260806 06:36:04.093362 5804 leader_election.cc:302] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 673943a7cd614424a522db1b12d7b4f0; no voters:
I20260806 06:36:04.093640 5804 leader_election.cc:288] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:04.093720 5811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:04.093942 5811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [term 1 LEADER]: Becoming Leader. State: Replica: 673943a7cd614424a522db1b12d7b4f0, State: Running, Role: LEADER
I20260806 06:36:04.093990 5804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:04.094060 5811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "673943a7cd614424a522db1b12d7b4f0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40747 } }
I20260806 06:36:04.094444 5812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "673943a7cd614424a522db1b12d7b4f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "673943a7cd614424a522db1b12d7b4f0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40747 } } }
I20260806 06:36:04.094560 5812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:04.094929 5813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 673943a7cd614424a522db1b12d7b4f0. Latest consensus state: current_term: 1 leader_uuid: "673943a7cd614424a522db1b12d7b4f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "673943a7cd614424a522db1b12d7b4f0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40747 } } }
I20260806 06:36:04.095358 5813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0 [sys.catalog]: This master's current role is: LEADER
W20260806 06:36:04.096220 5836 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:36:04.096284 5836 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:36:04.096330 5821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:04.096488 5821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:04.098362 5821 catalog_manager.cc:1347] Generated new cluster ID: e8cdad5346104c638b6776df2a0fdcdb
I20260806 06:36:04.098428 5821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:36:04.110220 5821 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:04.111018 5821 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:04.121812 5821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 673943a7cd614424a522db1b12d7b4f0: Generated new TSK 0
I20260806 06:36:04.122058 5821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:36:04.171531 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:04.129071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39972 (local address 127.29.142.190:40747)
0806 06:36:04.129137 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:04.129139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:04.129255 (+ 116us) server_negotiation.cc:407] Connection header received
0806 06:36:04.129320 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:04.129321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:04.129334 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:04.129423 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:36:04.129552 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:04.130384 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:04.131235 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:04.131556 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:04.131636 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:04.131815 (+ 179us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:36:04.171240 (+ 39425us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 06:36:04.171453 (+ 213us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39972: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 06:36:04.171628 5809 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39972: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 06:36:04.171672 5847 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:04.128976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:40747 (local address 127.0.0.1:39972)
0806 06:36:04.129184 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:04.129194 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:04.129252 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:04.129432 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:04.129433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:04.129435 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:36:04.129489 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:04.129491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:04.130473 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:04.130474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:04.131142 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:04.131146 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:04.131662 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:04.131663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:04.131706 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:04.131711 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:36:04.171436 (+ 39725us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 06:36:04.171552 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.142.190:40747: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":116,"threads_started":1}
I20260806 06:36:04.172089 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 5711
2026-08-06T06:36:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (313 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (97 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (95 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:36:05 dist-test-slave-3kk6 krb5kdc[5863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:36:05 dist-test-slave-3kk6 krb5kdc[5863](info): set up 2 sockets
Aug 06 06:36:05 dist-test-slave-3kk6 krb5kdc[5863](info): commencing operation
krb5kdc: starting...
W20260806 06:36:06.275660 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.883s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:36:06 dist-test-slave-3kk6 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:36:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:36:06.406064 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:41491
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:32959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:41491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:06.492738 5881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:06.492974 5881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:06.492997 5881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:06.494207 5881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:06.494258 5881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:06.494272 5881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:06.494285 5881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:06.494345 5881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:06.495975 5881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:32959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:41491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:41491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:06.496261 5881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:06.496517 5881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:06.501638 5887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:06.501624 5890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:06.501624 5895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:06.501947 5881 server_base.cc:1034] running on GCE node
I20260806 06:36:06.502228 5881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:06.502542 5881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:06.503732 5881 hybrid_clock.cc:648] HybridClock initialized: now 1785998166503721 us; error 52 us; skew 500 ppm
Aug 06 06:36:06 dist-test-slave-3kk6 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998166, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:06.506301 5881 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:36:06.506825 5881 webserver.cc:460] Webserver started at http://127.29.142.190:33287/ using document root <none> and password file <none>
I20260806 06:36:06.507062 5881 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:06.507134 5881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:06.507329 5881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:06.508458 5881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "89e6fd5776d841fd84541ebe5a76b5b0"
format_stamp: "Formatted at 2026-08-06 06:36:06 on dist-test-slave-3kk6"
server_key: "ccb6f9f73e099d4c9b6d30c69405baa5"
server_key_iv: "209090ac096fa87600694f4a12bd7b26"
server_key_version: "encryptionkey@0"
I20260806 06:36:06.508852 5881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "89e6fd5776d841fd84541ebe5a76b5b0"
format_stamp: "Formatted at 2026-08-06 06:36:06 on dist-test-slave-3kk6"
server_key: "ccb6f9f73e099d4c9b6d30c69405baa5"
server_key_iv: "209090ac096fa87600694f4a12bd7b26"
server_key_version: "encryptionkey@0"
I20260806 06:36:06.510303 5881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:36:06.511118 5903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:06.511358 5881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:36:06.511471 5881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "89e6fd5776d841fd84541ebe5a76b5b0"
format_stamp: "Formatted at 2026-08-06 06:36:06 on dist-test-slave-3kk6"
server_key: "ccb6f9f73e099d4c9b6d30c69405baa5"
server_key_iv: "209090ac096fa87600694f4a12bd7b26"
server_key_version: "encryptionkey@0"
I20260806 06:36:06.511557 5881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:06.533344 5881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:06.535584 5881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:06.535778 5881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:06.541028 5881 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:41491
I20260806 06:36:06.541064 5984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:41491 every 8 connection(s)
I20260806 06:36:06.541479 5881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:06.541955 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 5881
I20260806 06:36:06.542081 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:06.542279 30266 external_mini_cluster.cc:1428] Setting key e69cd3dd1423b766b1471aecbe2f908f
I20260806 06:36:06.542640 5985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:06.546483 5985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0: Bootstrap starting.
I20260806 06:36:06.547267 5985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:06.547585 5985 log.cc:824] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:06.548548 5985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0: No bootstrap required, opened a new log
Aug 06 06:36:06 dist-test-slave-3kk6 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:06.551095 5985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e6fd5776d841fd84541ebe5a76b5b0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41491 } }
I20260806 06:36:06.551316 5985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:06.551368 5985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89e6fd5776d841fd84541ebe5a76b5b0, State: Initialized, Role: FOLLOWER
I20260806 06:36:06.551522 5985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e6fd5776d841fd84541ebe5a76b5b0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41491 } }
I20260806 06:36:06.551613 5985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:06.551640 5985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:06.551721 5985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:06.552446 5985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e6fd5776d841fd84541ebe5a76b5b0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41491 } }
I20260806 06:36:06.552546 5985 leader_election.cc:302] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89e6fd5776d841fd84541ebe5a76b5b0; no voters:
I20260806 06:36:06.552783 5985 leader_election.cc:288] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:06.552819 5992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:06.552942 5992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [term 1 LEADER]: Becoming Leader. State: Replica: 89e6fd5776d841fd84541ebe5a76b5b0, State: Running, Role: LEADER
I20260806 06:36:06.553086 5985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:06.553093 5992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e6fd5776d841fd84541ebe5a76b5b0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41491 } }
I20260806 06:36:06.553597 5993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89e6fd5776d841fd84541ebe5a76b5b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e6fd5776d841fd84541ebe5a76b5b0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41491 } } }
I20260806 06:36:06.553871 5993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:06.553784 5997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89e6fd5776d841fd84541ebe5a76b5b0. Latest consensus state: current_term: 1 leader_uuid: "89e6fd5776d841fd84541ebe5a76b5b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e6fd5776d841fd84541ebe5a76b5b0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41491 } } }
I20260806 06:36:06.554266 5997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:06.554684 6009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:06.555052 5990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:06.542936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60098 (local address 127.29.142.190:41491)
0806 06:36:06.543130 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:06.543141 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:06.543176 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:36:06.543831 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:06.543842 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:06.543924 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:06.544094 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:06.544802 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:06.546175 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:06.547104 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:06.547769 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:06.547897 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:06.550863 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:06.550868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:06.550868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:06.550881 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:06.552878 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:06.553271 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:06.553274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:06.553275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:06.553377 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:06.553582 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:06.553583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:06.553583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:06.553911 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:06.554147 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:06.554687 (+ 540us) server_negotiation.cc:299] Negotiation successful
0806 06:36:06.554857 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260806 06:36:06.555299 6009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:06.556864 6009 catalog_manager.cc:1347] Generated new cluster ID: 26919853ad1f465d8d597a774447fc7c
I20260806 06:36:06.556922 6009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:36:06.565320 6009 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:06.566100 6009 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:06.577332 6009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89e6fd5776d841fd84541ebe5a76b5b0: Generated new TSK 0
I20260806 06:36:06.577587 6009 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:36:06.615482 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41491
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:32959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:06.701037 6022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:06.701292 6022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:06.701333 6022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:06.702520 6022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:06.702589 6022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:06.702632 6022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:36:06.704205 6022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:32959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:06.704480 6022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:06.704727 6022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:06.710064 6040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:06.710109 6035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:06.710120 6043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:06.710248 6022 server_base.cc:1034] running on GCE node
I20260806 06:36:06.710530 6022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:06.710873 6022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:06.712049 6022 hybrid_clock.cc:648] HybridClock initialized: now 1785998166712025 us; error 44 us; skew 500 ppm
Aug 06 06:36:06 dist-test-slave-3kk6 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998166, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:06.715224 6022 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:36:06.715708 6022 webserver.cc:460] Webserver started at http://127.29.142.129:46357/ using document root <none> and password file <none>
I20260806 06:36:06.715925 6022 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:06.715968 6022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:06.716058 6022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:06.717111 6022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "f85f15931e834181b6a3b3e94528cef4"
format_stamp: "Formatted at 2026-08-06 06:36:06 on dist-test-slave-3kk6"
server_key: "7c2d0b8e6a2b0fc128b4aa63ffe0791f"
server_key_iv: "3b36fb2b888fb25d7ddb74c444a2434e"
server_key_version: "encryptionkey@0"
I20260806 06:36:06.717434 6022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "f85f15931e834181b6a3b3e94528cef4"
format_stamp: "Formatted at 2026-08-06 06:36:06 on dist-test-slave-3kk6"
server_key: "7c2d0b8e6a2b0fc128b4aa63ffe0791f"
server_key_iv: "3b36fb2b888fb25d7ddb74c444a2434e"
server_key_version: "encryptionkey@0"
I20260806 06:36:06.718830 6022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:06.719604 6053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:06.719777 6022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:06.719877 6022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "f85f15931e834181b6a3b3e94528cef4"
format_stamp: "Formatted at 2026-08-06 06:36:06 on dist-test-slave-3kk6"
server_key: "7c2d0b8e6a2b0fc128b4aa63ffe0791f"
server_key_iv: "3b36fb2b888fb25d7ddb74c444a2434e"
server_key_version: "encryptionkey@0"
I20260806 06:36:06.719969 6022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:06.732781 6022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:06.735095 6022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:06.735304 6022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:06.735589 6022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:06.735944 6022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:06.736003 6022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:06.736040 6022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:06.736088 6022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:06.742633 6022 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:34901
I20260806 06:36:06.742738 6218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:34901 every 8 connection(s)
I20260806 06:36:06.743175 6022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:36:06 dist-test-slave-3kk6 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998166, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:06.751912 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 6022
I20260806 06:36:06.752044 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:36:06.752233 30266 external_mini_cluster.cc:1428] Setting key 560721a4400125eb029e8049d5ca5335
I20260806 06:36:06.753477 5990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:06.744075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:33065 (local address 127.29.142.190:41491)
0806 06:36:06.744177 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:06.744180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:06.744369 (+ 189us) server_negotiation.cc:407] Connection header received
0806 06:36:06.745394 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:06.745396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:06.745407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:06.745477 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:06.746162 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:06.746717 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:06.747760 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:06.747970 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:06.748030 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:06.750407 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:06.750409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:06.750409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:06.750417 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:06.751865 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:06.752373 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:06.752375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:06.752375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:06.752438 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:06.752711 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:06.752712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:06.752713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:06.752888 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:06.752952 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:06.753267 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 06:36:06.753385 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 06:36:06.753677 6223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:06.744030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41491 (local address 127.29.142.129:33065)
0806 06:36:06.744228 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:06.744270 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:06.745265 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:06.745539 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:06.745543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:06.745844 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:06.746056 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:06.746061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:06.746809 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:06.746810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:06.747669 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:06.747672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:06.748055 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:06.748056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:06.748165 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:06.748709 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:06.748719 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:06.750303 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:06.751957 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:06.751964 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:06.751967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:06.752263 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:06.752515 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:06.752516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:06.752516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:06.752630 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:06.752955 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:06.752963 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:06.753115 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 06:36:06.753231 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 06:36:06.753413 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
I20260806 06:36:06.754228 6219 heartbeater.cc:347] Connected to a master server at 127.29.142.190:41491
I20260806 06:36:06.754348 6219 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:06.754559 6219 heartbeater.cc:511] Master 127.29.142.190:41491 requested a full tablet report, sending...
I20260806 06:36:06.755190 5930 ts_manager.cc:194] Registered new tserver with Master: f85f15931e834181b6a3b3e94528cef4 (127.29.142.129:34901)
I20260806 06:36:06.755823 5930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:33065
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:36:06.779961 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41491
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:32959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:06.865401 6227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:06.865655 6227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:06.865694 6227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:06.866886 6227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:06.866961 6227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:06.867007 6227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:36:06.868554 6227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:32959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:06.868845 6227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:06.869099 6227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:06.874464 6238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:06.874464 6235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:06.874531 6236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:06.874752 6227 server_base.cc:1034] running on GCE node
I20260806 06:36:06.875052 6227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:06.875341 6227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:06.876507 6227 hybrid_clock.cc:648] HybridClock initialized: now 1785998166876497 us; error 52 us; skew 500 ppm
Aug 06 06:36:06 dist-test-slave-3kk6 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998166, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:06.879192 6227 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:36:06.879676 6227 webserver.cc:460] Webserver started at http://127.29.142.130:36155/ using document root <none> and password file <none>
I20260806 06:36:06.879925 6227 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:06.880000 6227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:06.880162 6227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:06.881275 6227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "38cfd8535bd74459bdc948ee4aab7987"
format_stamp: "Formatted at 2026-08-06 06:36:06 on dist-test-slave-3kk6"
server_key: "ff3c3dc1975069e645d2fb9b9e774207"
server_key_iv: "f20a0c900bdfb046edc11a777585efd3"
server_key_version: "encryptionkey@0"
I20260806 06:36:06.881680 6227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "38cfd8535bd74459bdc948ee4aab7987"
format_stamp: "Formatted at 2026-08-06 06:36:06 on dist-test-slave-3kk6"
server_key: "ff3c3dc1975069e645d2fb9b9e774207"
server_key_iv: "f20a0c900bdfb046edc11a777585efd3"
server_key_version: "encryptionkey@0"
I20260806 06:36:06.883168 6227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:36:06.883919 6247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:06.884094 6227 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:36:06.884207 6227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "38cfd8535bd74459bdc948ee4aab7987"
format_stamp: "Formatted at 2026-08-06 06:36:06 on dist-test-slave-3kk6"
server_key: "ff3c3dc1975069e645d2fb9b9e774207"
server_key_iv: "f20a0c900bdfb046edc11a777585efd3"
server_key_version: "encryptionkey@0"
I20260806 06:36:06.884286 6227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:06.912104 6227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:06.914201 6227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:06.914448 6227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:06.914817 6227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:06.915179 6227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:06.915237 6227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:06.915283 6227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:06.915310 6227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:06.921703 6227 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:33659
I20260806 06:36:06.921746 6403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:33659 every 8 connection(s)
I20260806 06:36:06.922117 6227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:36:06.926049 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 6227
I20260806 06:36:06.926137 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:36:06.926352 30266 external_mini_cluster.cc:1428] Setting key d51617ebbd7a43cc6ff8d1b1b45d682d
Aug 06 06:36:06 dist-test-slave-3kk6 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998166, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:06.933086 5990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:06.923050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:53193 (local address 127.29.142.190:41491)
0806 06:36:06.923149 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:06.923151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:06.923345 (+ 194us) server_negotiation.cc:407] Connection header received
0806 06:36:06.924532 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:06.924534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:06.924548 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:06.924610 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:06.925314 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:06.925836 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:06.927224 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:06.927438 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:06.927516 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:06.929934 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:06.929935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:06.929936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:06.929945 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:06.931537 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:06.932011 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:06.932012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:06.932013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:06.932083 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:06.932363 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:06.932364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:06.932364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:06.932501 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:06.932579 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:06.932896 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 06:36:06.932987 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:36:06.933311 6406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:06.923000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41491 (local address 127.29.142.130:53193)
0806 06:36:06.923209 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:06.923248 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:06.924402 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:06.924687 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:06.924696 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:06.924981 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:06.925188 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:06.925192 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:06.925944 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:06.925945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:06.927118 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:06.927122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:06.927540 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:06.927540 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:06.927685 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:06.928303 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:06.928313 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:06.929814 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:06.931661 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:06.931669 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:06.931672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:06.931925 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:06.932163 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:06.932164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:06.932164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:06.932284 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:06.932583 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:06.932590 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:06.932741 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 06:36:06.932857 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 06:36:06.933036 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":100,"threads_started":1}
I20260806 06:36:06.933842 6404 heartbeater.cc:347] Connected to a master server at 127.29.142.190:41491
I20260806 06:36:06.933969 6404 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:06.934195 6404 heartbeater.cc:511] Master 127.29.142.190:41491 requested a full tablet report, sending...
I20260806 06:36:06.934725 5930 ts_manager.cc:194] Registered new tserver with Master: 38cfd8535bd74459bdc948ee4aab7987 (127.29.142.130:33659)
I20260806 06:36:06.935119 5930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:53193
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:36:06.955276 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41491
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:32959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:07.042143 6410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:07.042390 6410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:07.042416 6410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:07.043659 6410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:07.043715 6410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:07.043742 6410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:36:07.045256 6410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:32959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:07.045475 6410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:07.045665 6410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:07.051136 6418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:07.051139 6415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:07.051142 6416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:07.051437 6410 server_base.cc:1034] running on GCE node
I20260806 06:36:07.051756 6410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:07.052053 6410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:07.053236 6410 hybrid_clock.cc:648] HybridClock initialized: now 1785998167053291 us; error 103 us; skew 500 ppm
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998167, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:07.056452 6410 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:36:07.056957 6410 webserver.cc:460] Webserver started at http://127.29.142.131:39791/ using document root <none> and password file <none>
I20260806 06:36:07.057219 6410 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:07.057292 6410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:07.057454 6410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:07.058590 6410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "7ccff6ceea6f4ee7a33c0b6e5a10f179"
format_stamp: "Formatted at 2026-08-06 06:36:07 on dist-test-slave-3kk6"
server_key: "d31024e57a4bef6a5f5772e587fec04d"
server_key_iv: "52582b6137a458b90cc8bb38d19a9efc"
server_key_version: "encryptionkey@0"
I20260806 06:36:07.059056 6410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "7ccff6ceea6f4ee7a33c0b6e5a10f179"
format_stamp: "Formatted at 2026-08-06 06:36:07 on dist-test-slave-3kk6"
server_key: "d31024e57a4bef6a5f5772e587fec04d"
server_key_iv: "52582b6137a458b90cc8bb38d19a9efc"
server_key_version: "encryptionkey@0"
I20260806 06:36:07.060554 6410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:36:07.061389 6430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:07.061583 6410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:07.061697 6410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "7ccff6ceea6f4ee7a33c0b6e5a10f179"
format_stamp: "Formatted at 2026-08-06 06:36:07 on dist-test-slave-3kk6"
server_key: "d31024e57a4bef6a5f5772e587fec04d"
server_key_iv: "52582b6137a458b90cc8bb38d19a9efc"
server_key_version: "encryptionkey@0"
I20260806 06:36:07.061781 6410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:07.073474 6410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:07.076087 6410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:07.076280 6410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:07.076553 6410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:07.076912 6410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:07.076972 6410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:07.077008 6410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:07.077054 6410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:07.083630 6410 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:39533
I20260806 06:36:07.083670 6587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:39533 every 8 connection(s)
I20260806 06:36:07.084051 6410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998167, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:07.091213 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 6410
I20260806 06:36:07.091336 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:36:07.091552 30266 external_mini_cluster.cc:1428] Setting key f93a0ecf5061c540757d58cfadd4ea67
I20260806 06:36:07.094513 5990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:07.084888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:52293 (local address 127.29.142.190:41491)
0806 06:36:07.084975 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:07.084978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:07.085143 (+ 165us) server_negotiation.cc:407] Connection header received
0806 06:36:07.086143 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:07.086145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:07.086157 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:07.086219 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:07.086955 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:07.087538 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:07.088693 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:07.088881 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:07.088938 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:07.091426 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:07.091427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:07.091428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:07.091436 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:07.093018 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:07.093477 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:07.093478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:07.093479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:07.093544 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:07.093816 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:07.093817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:07.093817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:07.093953 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:07.094035 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:07.094324 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 06:36:07.094411 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:36:07.094761 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:07.084826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41491 (local address 127.29.142.131:52293)
0806 06:36:07.085003 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:07.085045 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:07.086013 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:07.086285 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:07.086289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:07.086566 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:07.086858 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:07.086863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:07.087623 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:07.087624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:07.088608 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:07.088611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:07.088959 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:07.088960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:07.089062 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:07.089685 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:07.089694 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:07.091301 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:07.093127 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:07.093136 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:07.093139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:07.093392 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:07.093615 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:07.093616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:07.093617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:07.093742 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:07.094027 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:07.094034 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:07.094185 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 06:36:07.094303 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 06:36:07.094486 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":109,"threads_started":1}
I20260806 06:36:07.095314 6588 heartbeater.cc:347] Connected to a master server at 127.29.142.190:41491
I20260806 06:36:07.095412 6588 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:07.095563 6588 heartbeater.cc:511] Master 127.29.142.190:41491 requested a full tablet report, sending...
I20260806 06:36:07.095925 5930 ts_manager.cc:194] Registered new tserver with Master: 7ccff6ceea6f4ee7a33c0b6e5a10f179 (127.29.142.131:39533)
I20260806 06:36:07.096392 5930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:52293
I20260806 06:36:07.103402 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/test-admin.keytab.
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:07.119834 30266 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 06:36:07.121783 6597 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:07.120835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51952 (local address 127.29.142.129:34901)
0806 06:36:07.121031 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:07.121038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:07.121048 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:36:07.121086 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:07.121087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:07.121101 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:07.121187 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:07.121694 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51952: BlockingRecv error: recv got EOF from 127.0.0.1:51952 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
W20260806 06:36:07.121958 30266 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/krb5cc' of type FILE
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:07.123067 30266 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:07.124024 30266 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.129@KRBTEST.COM
I20260806 06:36:07.132375 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:07.124548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51954 (local address 127.29.142.129:34901)
0806 06:36:07.124619 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:07.124621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:07.124634 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:36:07.124732 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:07.124732 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:07.124752 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:07.124816 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:07.125241 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:07.125969 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:07.126781 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:07.127127 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:07.127216 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:07.129527 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:07.129538 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:07.129543 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:07.129550 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:07.131102 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:07.131490 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:07.131494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:07.131495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:07.131558 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:07.131775 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:07.131776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:07.131777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:07.131930 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:07.132098 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:07.132158 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 06:36:07.132243 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 06:36:07.134351 6597 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:07.133697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51962 (local address 127.29.142.129:34901)
0806 06:36:07.133743 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:07.133745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:07.133773 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:36:07.133840 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:07.133840 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:07.133850 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:07.133897 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:07.134295 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51962: BlockingRecv error: recv got EOF from 127.0.0.1:51962 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:36:07.134482 30266 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/krb5cc)
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:07.135596 30266 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:07.136550 30266 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.129@KRBTEST.COM
I20260806 06:36:07.143667 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:07.137094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.29.142.129:34901)
0806 06:36:07.137150 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:07.137152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:07.137177 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:36:07.137240 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:07.137240 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:07.137248 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:07.137289 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:07.137622 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:07.138124 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:07.138888 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:07.139086 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:07.139141 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:07.141226 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:07.141228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:07.141228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:07.141235 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:07.142530 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:07.142935 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:07.142935 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:07.142936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:07.142988 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:07.143160 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:07.143160 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:07.143161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:07.143249 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:07.143326 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:07.143469 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 06:36:07.143532 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 06:36:07.145254 6597 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:07.144793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51978 (local address 127.29.142.129:34901)
0806 06:36:07.144855 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:07.144857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:07.144885 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:36:07.144902 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:07.144902 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:07.144910 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:07.144948 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:07.145192 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51978: BlockingRecv error: recv got EOF from 127.0.0.1:51978 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 06:36:07.145390 30266 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998137760399-30266-0/krb5kdc/krb5cc' of type FILE
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:07.146397 30266 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[5863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:07.147348 30266 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[5863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.129@KRBTEST.COM
I20260806 06:36:07.154423 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:07.147902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51990 (local address 127.29.142.129:34901)
0806 06:36:07.147955 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:07.147957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:07.147964 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:36:07.148029 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:07.148029 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:07.148037 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:07.148084 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:07.148423 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:07.148882 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:07.149518 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:07.149690 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:07.149739 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:07.151732 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:07.151733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:07.151734 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:07.151740 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:07.153314 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:07.153615 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:07.153616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:07.153616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:07.153732 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:07.153957 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:07.153958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:07.153958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:07.154063 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:07.154139 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:07.154261 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 06:36:07.154319 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:36:07.155143 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 6022
I20260806 06:36:07.171968 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 6227
I20260806 06:36:07.179649 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 6410
I20260806 06:36:07.189656 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 5881
2026-08-06T06:36:07Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2824 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[6615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[6615](info): set up 2 sockets
Aug 06 06:36:07 dist-test-slave-3kk6 krb5kdc[6615](info): commencing operation
krb5kdc: starting...
W20260806 06:36:08.413532 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:36:08 dist-test-slave-3kk6 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:36:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.190.keytab.
Entry for principal oryx/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.190.keytab.
I20260806 06:36:08.543808 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:39715
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.190.keytab
--principal=oryx/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:39715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:08.631855 6633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:08.632087 6633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:08.632113 6633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:08.633296 6633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:08.633345 6633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:08.633360 6633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:08.633400 6633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:08.633432 6633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:08.634974 6633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:39715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.190.keytab
--principal=oryx/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:39715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:08.635251 6633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:08.635504 6633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:08.640658 6640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:08.640635 6639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:08.640610 6642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:08.640846 6633 server_base.cc:1034] running on GCE node
I20260806 06:36:08.641098 6633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:08.641466 6633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:08.642627 6633 hybrid_clock.cc:648] HybridClock initialized: now 1785998168642617 us; error 30 us; skew 500 ppm
Aug 06 06:36:08 dist-test-slave-3kk6 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998168, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:08.645252 6633 init.cc:377] Logged in from keytab as oryx/127.29.142.190@KRBTEST.COM (short username oryx)
I20260806 06:36:08.645787 6633 webserver.cc:460] Webserver started at http://127.29.142.190:36847/ using document root <none> and password file <none>
I20260806 06:36:08.646027 6633 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:08.646098 6633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:08.646263 6633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:08.647490 6633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "02bc53d761e34b5385e5312657e82269"
format_stamp: "Formatted at 2026-08-06 06:36:08 on dist-test-slave-3kk6"
server_key: "bb29cc01f3336b568e703bdd7e8c8be8"
server_key_iv: "58c53479aeb49a4ff2a68fa4bb4f138a"
server_key_version: "encryptionkey@0"
I20260806 06:36:08.647907 6633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "02bc53d761e34b5385e5312657e82269"
format_stamp: "Formatted at 2026-08-06 06:36:08 on dist-test-slave-3kk6"
server_key: "bb29cc01f3336b568e703bdd7e8c8be8"
server_key_iv: "58c53479aeb49a4ff2a68fa4bb4f138a"
server_key_version: "encryptionkey@0"
I20260806 06:36:08.649466 6633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:36:08.650274 6654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:08.650465 6633 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:08.650580 6633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "02bc53d761e34b5385e5312657e82269"
format_stamp: "Formatted at 2026-08-06 06:36:08 on dist-test-slave-3kk6"
server_key: "bb29cc01f3336b568e703bdd7e8c8be8"
server_key_iv: "58c53479aeb49a4ff2a68fa4bb4f138a"
server_key_version: "encryptionkey@0"
I20260806 06:36:08.650691 6633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:08.676733 6633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:08.679047 6633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:08.679230 6633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:08.683564 6633 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:39715
I20260806 06:36:08.683604 6737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:39715 every 8 connection(s)
I20260806 06:36:08.683993 6633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:08.685051 6739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:08.687608 6739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269: Bootstrap starting.
I20260806 06:36:08.688252 6739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:08.688565 6739 log.cc:824] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:08.689320 6739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269: No bootstrap required, opened a new log
I20260806 06:36:08.689961 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 6633
I20260806 06:36:08.690078 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:08.690279 30266 external_mini_cluster.cc:1428] Setting key 9103e62bd919417ca45a11f754a6a1c2
I20260806 06:36:08.692198 6739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } }
I20260806 06:36:08.692399 6739 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:08.692446 6739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02bc53d761e34b5385e5312657e82269, State: Initialized, Role: FOLLOWER
I20260806 06:36:08.692589 6739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } }
I20260806 06:36:08.692683 6739 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:08.692723 6739 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:08.692777 6739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:08.693490 6739 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } }
I20260806 06:36:08.693586 6739 leader_election.cc:302] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02bc53d761e34b5385e5312657e82269; no voters:
I20260806 06:36:08.693807 6739 leader_election.cc:288] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:08.693840 6745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:08.693987 6745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 1 LEADER]: Becoming Leader. State: Replica: 02bc53d761e34b5385e5312657e82269, State: Running, Role: LEADER
I20260806 06:36:08.694074 6745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } }
I20260806 06:36:08.694130 6739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:08.694412 6745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } }
I20260806 06:36:08.694486 6745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:08.694626 6745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02bc53d761e34b5385e5312657e82269. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } }
I20260806 06:36:08.694736 6745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:08.695034 6751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:08.695601 6751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:08.697271 6751 catalog_manager.cc:1347] Generated new cluster ID: 441134192b9a48d5b70cc90a71af07bb
I20260806 06:36:08.697329 6751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:36:08 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
I20260806 06:36:08.703528 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:08.690999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42278 (local address 127.29.142.190:39715)
0806 06:36:08.691192 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:08.691200 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:08.691227 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:36:08.692123 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:08.692131 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:08.692184 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:08.692316 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:08.693162 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:08.694387 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:08.695573 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:08.696250 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:08.696402 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:08.699439 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:08.699443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:08.699443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:08.699456 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:08.701658 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:08.702077 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:08.702080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:08.702081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:08.702210 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:08.702430 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:08.702431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:08.702432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:08.702806 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:08.702995 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:08.703123 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 06:36:08.703304 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260806 06:36:08.718638 6751 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:08.719450 6751 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:08.722265 6751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269: Generated new TSK 0
I20260806 06:36:08.722467 6751 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab.
Entry for principal oryx/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab.
I20260806 06:36:08.754033 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39715
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab
--principal=oryx/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:08.841624 6774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:08.841863 6774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:08.841888 6774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:08.843114 6774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:08.843161 6774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:08.843217 6774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:36:08.844715 6774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab
--principal=oryx/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:08.844977 6774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:08.845230 6774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:08.850524 6790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:08.850497 6788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:08.850759 6774 server_base.cc:1034] running on GCE node
W20260806 06:36:08.850791 6787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:08.851094 6774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:08.851357 6774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:08.852519 6774 hybrid_clock.cc:648] HybridClock initialized: now 1785998168852500 us; error 38 us; skew 500 ppm
Aug 06 06:36:08 dist-test-slave-3kk6 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998168, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:08.855383 6774 init.cc:377] Logged in from keytab as oryx/127.29.142.129@KRBTEST.COM (short username oryx)
I20260806 06:36:08.855919 6774 webserver.cc:460] Webserver started at http://127.29.142.129:39427/ using document root <none> and password file <none>
I20260806 06:36:08.856159 6774 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:08.856227 6774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:08.856391 6774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:08.857501 6774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "523ec1cf6a9e482a8fa0874e25825d9b"
format_stamp: "Formatted at 2026-08-06 06:36:08 on dist-test-slave-3kk6"
server_key: "1beda9f46319030ffbcc51fc1a21e88d"
server_key_iv: "38562232daa95d5b330ddbc75df7bd33"
server_key_version: "encryptionkey@0"
I20260806 06:36:08.857892 6774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "523ec1cf6a9e482a8fa0874e25825d9b"
format_stamp: "Formatted at 2026-08-06 06:36:08 on dist-test-slave-3kk6"
server_key: "1beda9f46319030ffbcc51fc1a21e88d"
server_key_iv: "38562232daa95d5b330ddbc75df7bd33"
server_key_version: "encryptionkey@0"
I20260806 06:36:08.859401 6774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:08.860222 6798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:08.860392 6774 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:08.860496 6774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "523ec1cf6a9e482a8fa0874e25825d9b"
format_stamp: "Formatted at 2026-08-06 06:36:08 on dist-test-slave-3kk6"
server_key: "1beda9f46319030ffbcc51fc1a21e88d"
server_key_iv: "38562232daa95d5b330ddbc75df7bd33"
server_key_version: "encryptionkey@0"
I20260806 06:36:08.860571 6774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:08.877789 6774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:08.879966 6774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:08.880155 6774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:08.880419 6774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:08.880746 6774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:08.880798 6774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:08.880862 6774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:08.880894 6774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:08.886767 6774 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:41849
I20260806 06:36:08.886855 6957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:41849 every 8 connection(s)
I20260806 06:36:08.887207 6774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:36:08.890388 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 6774
I20260806 06:36:08.890528 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:36:08.890787 30266 external_mini_cluster.cc:1428] Setting key 31c783de49332925d1e67bd6300bc2a7
Aug 06 06:36:08 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998168, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
I20260806 06:36:08.898048 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:08.888015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:51147 (local address 127.29.142.190:39715)
0806 06:36:08.888078 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:08.888080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:08.888260 (+ 180us) server_negotiation.cc:407] Connection header received
0806 06:36:08.888799 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:08.888800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:08.888810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:08.888857 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:08.889832 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:08.890377 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:08.891676 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:08.891899 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:08.891958 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:08.894730 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:08.894731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:08.894731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:08.894738 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:08.896550 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:08.896989 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:08.896991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:08.896991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:08.897066 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:08.897310 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:08.897312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:08.897312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:08.897491 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:08.897575 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:08.897821 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 06:36:08.897936 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:36:08.898073 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:08.887951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.129:51147)
0806 06:36:08.888119 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:08.888164 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:08.888674 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:08.888915 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:08.888918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:08.889522 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:08.889727 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:08.889732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:08.890477 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:08.890478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:08.891569 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:08.891572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:08.891990 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:08.891992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:08.892158 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:08.892730 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:08.892741 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:08.894565 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:08.896653 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:08.896660 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:08.896662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:08.896911 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:08.897141 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:08.897141 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:08.897142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:08.897244 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:08.897561 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:08.897566 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:08.897685 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 06:36:08.897769 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:36:08.897900 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260806 06:36:08.898622 6958 heartbeater.cc:347] Connected to a master server at 127.29.142.190:39715
I20260806 06:36:08.898773 6958 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:08.898975 6958 heartbeater.cc:511] Master 127.29.142.190:39715 requested a full tablet report, sending...
I20260806 06:36:08.899530 6682 ts_manager.cc:194] Registered new tserver with Master: 523ec1cf6a9e482a8fa0874e25825d9b (127.29.142.129:41849)
I20260806 06:36:08.900146 6682 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:51147
WARNING: no policy specified for oryx/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab.
Entry for principal oryx/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab.
I20260806 06:36:08.919695 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39715
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab
--principal=oryx/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:09.004032 6964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:09.004311 6964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:09.004369 6964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:09.005560 6964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:09.005637 6964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:09.005683 6964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:36:09.007217 6964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab
--principal=oryx/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:09.007511 6964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:09.007761 6964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:09.013002 6970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:09.012986 6975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:09.012987 6971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:09.013123 6964 server_base.cc:1034] running on GCE node
I20260806 06:36:09.013499 6964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:09.013746 6964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:09.014906 6964 hybrid_clock.cc:648] HybridClock initialized: now 1785998169014889 us; error 32 us; skew 500 ppm
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:09.017432 6964 init.cc:377] Logged in from keytab as oryx/127.29.142.130@KRBTEST.COM (short username oryx)
I20260806 06:36:09.017927 6964 webserver.cc:460] Webserver started at http://127.29.142.130:43341/ using document root <none> and password file <none>
I20260806 06:36:09.018160 6964 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:09.018226 6964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:09.018380 6964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:09.019552 6964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "270b539b1e414d63952194c491296c18"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "4bc86c4ac4a752527942ca6b1f1e9d20"
server_key_iv: "ec2531af528d00db74e596e3dea7c7d1"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.019940 6964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "270b539b1e414d63952194c491296c18"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "4bc86c4ac4a752527942ca6b1f1e9d20"
server_key_iv: "ec2531af528d00db74e596e3dea7c7d1"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.021382 6964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:36:09.022186 6985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.022385 6964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:09.022495 6964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "270b539b1e414d63952194c491296c18"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "4bc86c4ac4a752527942ca6b1f1e9d20"
server_key_iv: "ec2531af528d00db74e596e3dea7c7d1"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.022571 6964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:09.035440 6964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:09.039530 6964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:09.039718 6964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:09.040022 6964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:09.040464 6964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:09.040519 6964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.040553 6964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:09.040580 6964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.047173 6964 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:39021
I20260806 06:36:09.047241 7144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:39021 every 8 connection(s)
I20260806 06:36:09.047581 6964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
I20260806 06:36:09.055454 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 6964
I20260806 06:36:09.055573 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:36:09.055774 30266 external_mini_cluster.cc:1428] Setting key 61e24660ee8d78785368e0413534b70a
I20260806 06:36:09.057315 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.048356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:53155 (local address 127.29.142.190:39715)
0806 06:36:09.048438 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.048441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.048626 (+ 185us) server_negotiation.cc:407] Connection header received
0806 06:36:09.049178 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.049179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.049189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.049245 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.050217 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.050814 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.051828 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.052009 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.052061 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.054306 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.054307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.054307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.054315 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.055755 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.056249 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.056250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.056251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.056317 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.056621 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.056621 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.056622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.056732 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.056784 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.057158 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.057247 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:36:09.057628 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.048310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.130:53155)
0806 06:36:09.048490 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.048529 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.049085 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.049304 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.049307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.049921 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.050123 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.050128 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.050902 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.050903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.051752 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.051754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.052077 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.052077 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.052179 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.052710 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.052718 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.054196 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.055853 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.055862 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.055865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.056186 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.056384 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.056384 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.056385 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.056540 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.056796 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.056803 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.057001 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.057130 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.057327 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260806 06:36:09.058095 7145 heartbeater.cc:347] Connected to a master server at 127.29.142.190:39715
I20260806 06:36:09.058220 7145 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:09.058454 7145 heartbeater.cc:511] Master 127.29.142.190:39715 requested a full tablet report, sending...
I20260806 06:36:09.058935 6682 ts_manager.cc:194] Registered new tserver with Master: 270b539b1e414d63952194c491296c18 (127.29.142.130:39021)
I20260806 06:36:09.059355 6682 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:53155
WARNING: no policy specified for oryx/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab.
Entry for principal oryx/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab.
I20260806 06:36:09.081652 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39715
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab
--principal=oryx/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:09.167361 7151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:09.167594 7151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:09.167619 7151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:09.168812 7151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:09.168862 7151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:09.168910 7151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:36:09.170431 7151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab
--principal=oryx/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:09.170720 7151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:09.170984 7151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:09.176154 7166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:09.176168 7162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:09.176199 7164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:09.176388 7151 server_base.cc:1034] running on GCE node
I20260806 06:36:09.176614 7151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:09.176872 7151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:09.178037 7151 hybrid_clock.cc:648] HybridClock initialized: now 1785998169178020 us; error 29 us; skew 500 ppm
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:09.180763 7151 init.cc:377] Logged in from keytab as oryx/127.29.142.131@KRBTEST.COM (short username oryx)
I20260806 06:36:09.181277 7151 webserver.cc:460] Webserver started at http://127.29.142.131:43533/ using document root <none> and password file <none>
I20260806 06:36:09.181532 7151 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:09.181598 7151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:09.181748 7151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:09.182932 7151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "4e615fbf098747529b61f02005270202"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "2c7e11a34dfff8e4aa7df16b0aac1314"
server_key_iv: "7f8aeb069b80f7937b90a81e75798dff"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.183320 7151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "4e615fbf098747529b61f02005270202"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "2c7e11a34dfff8e4aa7df16b0aac1314"
server_key_iv: "7f8aeb069b80f7937b90a81e75798dff"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.184760 7151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:36:09.185524 7174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.185696 7151 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:09.185807 7151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "4e615fbf098747529b61f02005270202"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "2c7e11a34dfff8e4aa7df16b0aac1314"
server_key_iv: "7f8aeb069b80f7937b90a81e75798dff"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.185889 7151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:09.198535 7151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:09.200800 7151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:09.200987 7151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:09.201339 7151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:09.201728 7151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:09.201785 7151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.201830 7151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:09.201866 7151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.208334 7151 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:34625
I20260806 06:36:09.208382 7350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:34625 every 8 connection(s)
I20260806 06:36:09.208756 7151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
I20260806 06:36:09.217180 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 7151
I20260806 06:36:09.217267 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:36:09.217473 30266 external_mini_cluster.cc:1428] Setting key 06543b8967d5d2ce8057db412086393e
I20260806 06:36:09.219004 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.209622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:32897 (local address 127.29.142.190:39715)
0806 06:36:09.209682 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.209685 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.209994 (+ 309us) server_negotiation.cc:407] Connection header received
0806 06:36:09.210488 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.210489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.210500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.210558 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.211626 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.212143 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.213167 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.213377 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.213449 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.215826 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.215828 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.215828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.215836 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.217227 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.217770 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.217772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.217772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.217850 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.218146 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.218147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.218147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.218303 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.218363 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.218854 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.218936 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:36:09.219130 7353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.209615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.131:32897)
0806 06:36:09.209859 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.209900 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.210401 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.210615 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.210618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.211309 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.211523 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.211537 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.212242 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.212243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.213067 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.213069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.213467 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.213468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.213576 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.214188 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.214198 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.215726 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.217349 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.217356 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.217359 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.217651 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.217934 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.217935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.217936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.218089 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.218370 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.218377 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.218569 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.218738 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.218929 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":128,"threads_started":1}
I20260806 06:36:09.219580 7351 heartbeater.cc:347] Connected to a master server at 127.29.142.190:39715
I20260806 06:36:09.219686 7351 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:09.219844 7351 heartbeater.cc:511] Master 127.29.142.190:39715 requested a full tablet report, sending...
I20260806 06:36:09.220206 6682 ts_manager.cc:194] Registered new tserver with Master: 4e615fbf098747529b61f02005270202 (127.29.142.131:34625)
I20260806 06:36:09.220557 6682 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:32897
I20260806 06:36:09.229614 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.142.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.189.keytab.
Entry for principal oryx/127.29.142.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.189.keytab.
I20260806 06:36:09.257571 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.189:41567
--webserver_interface=127.29.142.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.189.keytab
--principal=oryx/127.29.142.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:39715,127.29.142.189:41567
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:09.343528 7356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:09.343806 7356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:09.343858 7356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:09.345049 7356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:09.345126 7356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:09.345149 7356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:09.345172 7356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:09.345201 7356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:09.346791 7356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:39715,127.29.142.189:41567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.189.keytab
--principal=oryx/127.29.142.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.189:41567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.142.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
W20260806 06:36:09.346964 7356 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.29.142.190:39715,127.29.142.189:41567'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 06:36:09.347134 7356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:09.347386 7356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:09.352746 7367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:09.352890 7356 server_base.cc:1034] running on GCE node
W20260806 06:36:09.352732 7368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:09.352751 7370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:09.353227 7356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:09.353564 7356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:09.354739 7356 hybrid_clock.cc:648] HybridClock initialized: now 1785998169354723 us; error 41 us; skew 500 ppm
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:09.357353 7356 init.cc:377] Logged in from keytab as oryx/127.29.142.189@KRBTEST.COM (short username oryx)
I20260806 06:36:09.357899 7356 webserver.cc:460] Webserver started at http://127.29.142.189:41559/ using document root <none> and password file <none>
I20260806 06:36:09.358163 7356 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:09.358234 7356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:09.358386 7356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:09.359654 7356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data/instance:
uuid: "7619e678ceb947c49d8eafffa4b13272"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "1de2483bdfb66532b283be2262ac8c66"
server_key_iv: "90732bd3cf90fc75c822d0f3f6f4840c"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.360033 7356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/wal/instance:
uuid: "7619e678ceb947c49d8eafffa4b13272"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "1de2483bdfb66532b283be2262ac8c66"
server_key_iv: "90732bd3cf90fc75c822d0f3f6f4840c"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.361481 7356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:36:09.362193 7379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.362380 7356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:36:09.362484 7356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/wal
uuid: "7619e678ceb947c49d8eafffa4b13272"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "1de2483bdfb66532b283be2262ac8c66"
server_key_iv: "90732bd3cf90fc75c822d0f3f6f4840c"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.362560 7356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:09.372926 7356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:09.374989 7356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:09.375155 7356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:09.379760 7356 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.189:41567
I20260806 06:36:09.379796 7457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.189:41567 every 8 connection(s)
I20260806 06:36:09.380183 7356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
I20260806 06:36:09.381286 7458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:09.382993 7458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:36:09.383436 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 7356
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.189@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.142.189@KRBTEST.COM
I20260806 06:36:09.393507 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.383850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.29.142.190:39715)
0806 06:36:09.383929 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.383932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.384136 (+ 204us) server_negotiation.cc:407] Connection header received
0806 06:36:09.384818 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.384820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.384832 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.384900 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.385453 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.385968 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.387112 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.387415 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.387469 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.390173 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.390174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.390174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.390182 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.392030 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.392439 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.392440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.392441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.392509 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.392764 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.392766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.392766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.392948 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.393027 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.393305 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.393408 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:36:09.393745 7461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.383800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42284)
0806 06:36:09.384004 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.384049 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.384683 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.384933 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.384935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.385203 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.385359 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.385363 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.386068 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.386069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.387033 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.387036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.387484 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.387485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.387628 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.388292 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.388304 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.390081 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.392124 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.392131 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.392133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.392363 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.392587 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.392588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.392588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.392689 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.393030 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.393038 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.393172 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.393280 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.393446 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":122,"threads_started":1}
I20260806 06:36:09.394209 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.384103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54138 (local address 127.29.142.189:41567)
0806 06:36:09.384260 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.384265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.384301 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:36:09.384685 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.384694 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.384723 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.384817 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.385490 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.386491 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.387378 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.387780 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.388422 (+ 642us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.390838 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.390848 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.390849 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.390857 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.392679 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.393029 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.393033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.393033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.393096 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.393310 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.393311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.393312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.393498 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.393918 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.393972 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.394057 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":76,"threads_started":1}
I20260806 06:36:09.394533 7458 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.189@KRBTEST.COM for oryx/127.29.142.189@KRBTEST.COM
I20260806 06:36:09.400676 7461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.394862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:41567 (local address 127.0.0.1:54150)
0806 06:36:09.394913 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.394925 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.394971 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.395094 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.395095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.395254 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.395324 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.395327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.395925 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.395926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.396398 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.396399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.396716 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.396717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.396768 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.397209 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.397214 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.398381 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.399772 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.399772 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.399773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.399943 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.400152 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.400152 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.400153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.400202 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.400420 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.400420 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.400456 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.400506 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.400589 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:36:09.400772 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.394932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54150 (local address 127.29.142.189:41567)
0806 06:36:09.394971 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.394973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.394981 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:36:09.395023 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.395023 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.395035 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.395088 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.395387 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.395858 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.396464 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.396659 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.396709 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.398461 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.398462 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.398462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.398468 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.399699 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.400030 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.400031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.400031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.400087 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.400254 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.400255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.400255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.400361 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.400426 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.400521 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.400636 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":21}
I20260806 06:36:09.402269 7458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: Bootstrap starting.
I20260806 06:36:09.403296 7458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:09.403602 7458 log.cc:824] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:09.404544 7458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: No bootstrap required, opened a new log
I20260806 06:36:09.406083 7458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } }
I20260806 06:36:09.406242 7458 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:09.406299 7458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7619e678ceb947c49d8eafffa4b13272, State: Initialized, Role: FOLLOWER
I20260806 06:36:09.406438 7458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } }
I20260806 06:36:09.406831 7458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:09.406795 7468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } } }
I20260806 06:36:09.406903 7468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:36:09.408480 7481 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:36:09.408550 7481 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:36:09.410559 7356 master_runner.cc:428] Detected that this master 7619e678ceb947c49d8eafffa4b13272 is joining an existing cluster
I20260806 06:36:09.410631 7356 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.142.189:41567
I20260806 06:36:09.418831 7492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.411677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42298)
0806 06:36:09.411865 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.411878 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.411951 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.412150 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.412151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.412340 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.412454 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.412456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.413505 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.413506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.414198 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.414200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.414781 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.414782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.414847 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.415388 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.415393 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.415742 (+ 349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.417686 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.417687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.417687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.417892 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.418113 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.418114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.418114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.418171 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.418480 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.418482 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.418536 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.418589 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.418731 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":120,"threads_started":1}
I20260806 06:36:09.418851 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.411940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.29.142.190:39715)
0806 06:36:09.412011 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.412013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.412023 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:36:09.412051 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.412053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.412065 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.412148 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.412568 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.413397 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.414301 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.414617 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.414767 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.415840 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.415841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.415841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.415850 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.417604 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.417970 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.417972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.417972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.418042 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.418232 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.418233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.418234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.418386 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.418456 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.418639 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.418754 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:36:09.425539 7492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.411695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:41567 (local address 127.0.0.1:54166)
0806 06:36:09.418937 (+ 7242us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.418951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.419036 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.419193 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.419194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.419352 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.419403 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.419405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.420121 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.420122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.420911 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.420913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.421478 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.421479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.421550 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.422014 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.422020 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.422400 (+ 380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.424288 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.424289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.424289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.424551 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.424840 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.424841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.424841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.424932 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.425234 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.425235 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.425267 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.425331 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.425438 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7227,"spinlock_wait_cycles":10624,"thread_start_us":148,"threads_started":1}
I20260806 06:36:09.426165 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.411698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54166 (local address 127.29.142.189:41567)
0806 06:36:09.411749 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.411751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.419005 (+ 7254us) server_negotiation.cc:407] Connection header received
0806 06:36:09.419108 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.419108 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.419118 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.419144 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.419490 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.420045 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.421014 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.421353 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.421431 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.422497 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.422498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.422498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.422504 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.424189 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.424668 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.424670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.424670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.424749 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.425013 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.425014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.425015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.425152 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.425194 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.425403 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.425565 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998168, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.190@KRBTEST.COM for oryx/127.29.142.189@KRBTEST.COM
I20260806 06:36:09.434584 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.421593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54182 (local address 127.29.142.189:41567)
0806 06:36:09.426270 (+ 4677us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.426272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.426921 (+ 649us) server_negotiation.cc:407] Connection header received
0806 06:36:09.427025 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.427026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.427035 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.427077 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.427627 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.428078 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.429224 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.429398 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.429451 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.431863 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.431865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.431865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.431873 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.433282 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.433707 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.433708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.433709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.433771 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.433993 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.433993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.433994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.434109 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.434183 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.434385 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.434474 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4660,"thread_start_us":91,"threads_started":1}
I20260806 06:36:09.434603 7495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.421559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:41567 (local address 127.0.0.1:54182)
0806 06:36:09.426800 (+ 5241us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.426865 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.426952 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.427085 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.427088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.427471 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.427544 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.427547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.428154 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.428155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.429133 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.429137 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.429471 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.429472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.429579 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.430089 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.430095 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.431736 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.433392 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.433400 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.433403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.433628 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.433855 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.433856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.433857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.433923 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.434174 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.434183 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.434242 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.434355 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.434491 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5199,"thread_start_us":104,"threads_started":1}
I20260806 06:36:09.435333 6682 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.142.189:41567: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7619e678ceb947c49d8eafffa4b13272"
member_type: NON_VOTER
last_known_addr {
host: "127.29.142.189"
port: 41567
}
attrs {
promote: true
}
}
dest_uuid: "02bc53d761e34b5385e5312657e82269"
cas_config_opid_index: -1
I20260806 06:36:09.435802 6682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: NON_VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: true } }
I20260806 06:36:09.436326 7497 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7619e678ceb947c49d8eafffa4b13272 (127.29.142.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: NON_VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: true } } }
I20260806 06:36:09.436646 7420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:09.437501 7497 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.142.189:41567
I20260806 06:36:09.437652 7420 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 FOLLOWER]: Refusing update from remote peer 02bc53d761e34b5385e5312657e82269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 06:36:09.437892 7356 master.cc:537] Master@127.29.142.189:41567 shutting down...
I20260806 06:36:09.437997 7497 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: NON_VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:36:09.438791 6656 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 06:36:09.439185 7498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: NON_VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: true } } }
I20260806 06:36:09.439291 7498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: This master's current role is: LEADER
W20260806 06:36:09.439744 6656 consensus_peers.cc:487] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 -> Peer 7619e678ceb947c49d8eafffa4b13272 (127.29.142.189:41567): Couldn't send request to peer 7619e678ceb947c49d8eafffa4b13272. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 06:36:09.441185 7356 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:09.441293 7356 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:09.441350 7356 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: stopping tablet replica
I20260806 06:36:09.455349 7356 master.cc:552] Master@127.29.142.189:41567 shutdown complete.
I20260806 06:36:09.455423 7356 master_runner.cc:305] Clearing existing system tablet
I20260806 06:36:09.455569 7356 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:09.455624 7356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:09.456432 7356 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:36:09.457023 7502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.457147 7356 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:09.457209 7356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.457249 7356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/wal
uuid: "7619e678ceb947c49d8eafffa4b13272"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "1de2483bdfb66532b283be2262ac8c66"
server_key_iv: "90732bd3cf90fc75c822d0f3f6f4840c"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.458053 7356 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:09.459787 7356 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 06:36:09.459859 7356 log.cc:1196] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 06:36:09.460088 7356 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: Deleting consensus metadata
I20260806 06:36:09.460233 7356 master_runner.cc:315] Copying system tablet from 127.29.142.190:39715
I20260806 06:36:09.460903 7356 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: tablet copy: Beginning tablet copy session from remote peer at address 127.29.142.190:39715
I20260806 06:36:09.467891 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.461423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.29.142.190:39715)
0806 06:36:09.461477 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.461480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.461559 (+ 79us) server_negotiation.cc:407] Connection header received
0806 06:36:09.461634 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.461636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.461645 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.461735 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.462132 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.462644 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.463882 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.464099 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.464150 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.465261 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.465263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.465263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.465270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.466792 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.467174 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.467175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.467175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.467229 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.467422 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.467423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.467423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.467530 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.467582 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.467743 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.467808 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:36:09.467949 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.461321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42306)
0806 06:36:09.461471 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.461484 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.461568 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.461765 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.461766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.461984 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.462054 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.462056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.462815 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.462816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.463769 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.463772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.464165 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.464166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.464237 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.464743 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.464749 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.465169 (+ 420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.466874 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.466875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.466876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.467104 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.467296 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.467297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.467297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.467363 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.467590 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.467591 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.467641 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.467714 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.467828 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260806 06:36:09.468367 6709 tablet_copy_service.cc:140] P 02bc53d761e34b5385e5312657e82269: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7619e678ceb947c49d8eafffa4b13272 ({username='oryx', principal='oryx/127.29.142.189@KRBTEST.COM'} at 127.0.0.1:42306)
I20260806 06:36:09.468520 6709 tablet_copy_service.cc:161] P 02bc53d761e34b5385e5312657e82269: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7619e678ceb947c49d8eafffa4b13272 at {username='oryx', principal='oryx/127.29.142.189@KRBTEST.COM'} at 127.0.0.1:42306: session id = 7619e678ceb947c49d8eafffa4b13272-00000000000000000000000000000000
I20260806 06:36:09.469519 6709 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 06:36:09.470177 7356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 06:36:09.471343 7356 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: tablet copy: Starting download of 0 data blocks...
I20260806 06:36:09.471517 7356 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: tablet copy: Starting download of 1 WAL segments...
I20260806 06:36:09.472570 7356 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 06:36:09.473385 6709 tablet_copy_service.cc:342] P 02bc53d761e34b5385e5312657e82269: Request end of tablet copy session 7619e678ceb947c49d8eafffa4b13272-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.142.189@KRBTEST.COM'} at 127.0.0.1:42306
I20260806 06:36:09.473449 6709 tablet_copy_service.cc:433] P 02bc53d761e34b5385e5312657e82269: ending tablet copy session 7619e678ceb947c49d8eafffa4b13272-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7619e678ceb947c49d8eafffa4b13272
I20260806 06:36:09.476266 7356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:09.477501 7512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:09.477778 7356 server_base.cc:1034] running on GCE node
W20260806 06:36:09.477948 7515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:09.478307 7511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:09.478452 7356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:09.478624 7356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:09.479784 7356 hybrid_clock.cc:648] HybridClock initialized: now 1785998169479818 us; error 76 us; skew 500 ppm
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:09.481324 7356 init.cc:377] Logged in from keytab as oryx/127.29.142.189@KRBTEST.COM (short username oryx)
I20260806 06:36:09.481607 7356 webserver.cc:460] Webserver started at http://127.29.142.189:39427/ using document root <none> and password file <none>
I20260806 06:36:09.481727 7356 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:09.481767 7356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:09.482542 7356 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:09.483163 7524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.483336 7356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.483395 7356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/wal
uuid: "7619e678ceb947c49d8eafffa4b13272"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "1de2483bdfb66532b283be2262ac8c66"
server_key_iv: "90732bd3cf90fc75c822d0f3f6f4840c"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.483459 7356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:09.496454 7356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:09.498219 7356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:09.501690 7356 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.189:41567
I20260806 06:36:09.502065 7606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.189:41567 every 8 connection(s)
I20260806 06:36:09.502210 7356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
I20260806 06:36:09.503139 7607 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:36:09.503582 7607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: Bootstrap starting.
I20260806 06:36:09.506013 7607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:36:09.506259 7607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: Bootstrap complete.
I20260806 06:36:09.506471 7607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: NON_VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: true } }
I20260806 06:36:09.506561 7607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7619e678ceb947c49d8eafffa4b13272, State: Initialized, Role: LEARNER
I20260806 06:36:09.506613 7607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: NON_VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: true } }
I20260806 06:36:09.506848 7607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:09.506824 7610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: NON_VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: true } } }
I20260806 06:36:09.506907 7610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:36:09.508342 7622 catalog_manager.cc:1259] Loaded cluster ID: 441134192b9a48d5b70cc90a71af07bb
I20260806 06:36:09.508400 7622 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: loading cluster ID for follower catalog manager: success
I20260806 06:36:09.509351 7622 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: acquiring CA information for follower catalog manager: success
I20260806 06:36:09.509624 7622 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:36:09.553598 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.546699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54192 (local address 127.29.142.189:41567)
0806 06:36:09.547008 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.547010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.547018 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:36:09.547045 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.547046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.547056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.547141 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.547756 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.548443 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.549177 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.549390 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.549470 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.550819 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.550820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.550821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.550828 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.552316 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.552712 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.552713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.552713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.552788 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.553001 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.553002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.553002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.553118 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.553249 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.553361 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.553450 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":239,"threads_started":1}
WARNING: no policy specified for oryx/127.29.142.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.188.keytab.
Entry for principal oryx/127.29.142.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.188.keytab.
I20260806 06:36:09.581364 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.188:35751
--webserver_interface=127.29.142.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.188.keytab
--principal=oryx/127.29.142.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:39715,127.29.142.189:41567,127.29.142.188:35751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:09.667555 7626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:09.667824 7626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:09.667865 7626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:09.669037 7626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:09.669107 7626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:09.669132 7626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:09.669155 7626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:09.669191 7626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:09.671104 7626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:39715,127.29.142.189:41567,127.29.142.188:35751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.188.keytab
--principal=oryx/127.29.142.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.188:35751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.142.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:09.671423 7626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:09.671679 7626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:09.676813 7635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:09.676849 7633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:09.676847 7638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:09.677079 7626 server_base.cc:1034] running on GCE node
I20260806 06:36:09.677332 7626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:09.677608 7626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:09.678741 7626 hybrid_clock.cc:648] HybridClock initialized: now 1785998169678713 us; error 50 us; skew 500 ppm
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:09.681481 7626 init.cc:377] Logged in from keytab as oryx/127.29.142.188@KRBTEST.COM (short username oryx)
I20260806 06:36:09.681955 7626 webserver.cc:460] Webserver started at http://127.29.142.188:41047/ using document root <none> and password file <none>
I20260806 06:36:09.682160 7626 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:09.682201 7626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:09.682294 7626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:09.683559 7626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data/instance:
uuid: "9589ecac6b124e538c642accf1810167"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "0874e19551652b2a7be2214e9f43bfc9"
server_key_iv: "d7dd70a3800ddd4ff44fbdf50cb01f1e"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.683959 7626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/wal/instance:
uuid: "9589ecac6b124e538c642accf1810167"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "0874e19551652b2a7be2214e9f43bfc9"
server_key_iv: "d7dd70a3800ddd4ff44fbdf50cb01f1e"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.685482 7626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:36:09.686276 7647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.686452 7626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:36:09.686558 7626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/wal
uuid: "9589ecac6b124e538c642accf1810167"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "0874e19551652b2a7be2214e9f43bfc9"
server_key_iv: "d7dd70a3800ddd4ff44fbdf50cb01f1e"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.686633 7626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:09.701766 7626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:09.704511 7626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:09.704700 7626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:09.711931 7626 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.188:35751
I20260806 06:36:09.711974 7716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.188:35751 every 8 connection(s)
I20260806 06:36:09.712383 7626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
I20260806 06:36:09.713457 7717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:09.715246 7717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:36:09.717437 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 7626
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.188@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.142.188@KRBTEST.COM
I20260806 06:36:09.725663 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.716111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42310 (local address 127.29.142.190:39715)
0806 06:36:09.716178 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.716184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.716377 (+ 193us) server_negotiation.cc:407] Connection header received
0806 06:36:09.716831 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.716833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.716844 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.716913 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.717619 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.718255 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.719387 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.719581 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.719668 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.722196 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.722198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.722198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.722206 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.723948 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.724467 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.724468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.724469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.724537 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.724842 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.724844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.724844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.725001 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.725058 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.725471 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.725558 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:36:09.725947 7721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.716023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42310)
0806 06:36:09.716232 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.716288 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.716741 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.716986 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.716989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.717252 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.717464 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.717469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.718404 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.718404 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.719293 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.719296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.719713 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.719714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.719829 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.720413 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.720421 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.722069 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.724054 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.724062 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.724065 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.724381 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.724626 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.724627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.724628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.724759 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.725086 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.725095 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.725270 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.725444 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.725588 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":125,"threads_started":1}
I20260806 06:36:09.726676 7717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:36:09.727592 7722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.717961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.29.142.188:35751)
0806 06:36:09.718152 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.718157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.718176 (+ 19us) server_negotiation.cc:407] Connection header received
0806 06:36:09.718195 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.718196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.718220 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.718303 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.718984 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.719842 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.720733 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.721088 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.721159 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.724041 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.724050 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.724051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.724068 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.725954 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.726336 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.726339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.726339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.726402 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.726772 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.726773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.726773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.726982 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.727256 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.727352 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.727458 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.188@KRBTEST.COM for oryx/127.29.142.189@KRBTEST.COM
I20260806 06:36:09.733157 7721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.726981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:41567 (local address 127.0.0.1:54204)
0806 06:36:09.727030 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.727041 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.727079 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.727256 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.727257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.727410 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.727485 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.727487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.728289 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.728290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.728792 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.728793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.729190 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.729191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.729279 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.729701 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.729707 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.730956 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.732319 (+ 1363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.732320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.732320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.732490 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.732663 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.732664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.732664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.732714 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.732926 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.732927 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.732962 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.733014 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.733089 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:36:09.733165 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.727068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54204 (local address 127.29.142.189:41567)
0806 06:36:09.727136 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.727138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.727146 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:36:09.727175 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.727175 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.727193 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.727253 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.727567 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.728186 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.728878 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.729104 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.729190 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.731037 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.731038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.731038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.731046 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.732244 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.732555 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.732555 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.732556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.732608 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.732764 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.732765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.732765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.732875 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.732929 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.733031 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.733102 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:36:09.733734 7717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.188@KRBTEST.COM for oryx/127.29.142.188@KRBTEST.COM
I20260806 06:36:09.740454 7722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.734078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60508 (local address 127.29.142.188:35751)
0806 06:36:09.734115 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.734117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.734125 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:36:09.734168 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.734169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.734180 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.734226 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.734465 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.735030 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.735671 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.735909 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.735981 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.737777 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.737779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.737779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.737785 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.739235 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.739612 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.739613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.739613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.739676 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.739893 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.739894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.739894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.740012 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.740081 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.740225 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.740322 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:36:09.740504 7721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.734016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:35751 (local address 127.0.0.1:60508)
0806 06:36:09.734065 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.734076 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.734117 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.734226 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.734228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.734367 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.734396 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.734397 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.735110 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.735111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.735578 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.735580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.735990 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.735991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.736040 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.736478 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.736483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.737676 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.739358 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.739359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.739360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.739529 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.739739 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.739740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.739740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.739819 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.740085 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.740086 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.740123 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.740204 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.740372 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":82}
I20260806 06:36:09.742149 7717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: Bootstrap starting.
I20260806 06:36:09.743184 7717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:09.743492 7717 log.cc:824] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:09.744234 7717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: No bootstrap required, opened a new log
I20260806 06:36:09.745961 7717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } }
I20260806 06:36:09.746099 7717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:09.746124 7717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9589ecac6b124e538c642accf1810167, State: Initialized, Role: FOLLOWER
I20260806 06:36:09.746251 7717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } }
I20260806 06:36:09.746577 7717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:09.746580 7724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } } }
I20260806 06:36:09.746778 7724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:36:09.748823 7741 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:36:09.748890 7741 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:36:09.751384 7626 master_runner.cc:428] Detected that this master 9589ecac6b124e538c642accf1810167 is joining an existing cluster
I20260806 06:36:09.751456 7626 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.142.188:35751
I20260806 06:36:09.760141 7722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.752945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60516 (local address 127.29.142.188:35751)
0806 06:36:09.752987 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.752989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.753344 (+ 355us) server_negotiation.cc:407] Connection header received
0806 06:36:09.753424 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.753425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.753433 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.753460 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.754011 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.754813 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.755900 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.756185 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.756224 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.757496 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.757497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.757497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.757503 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.759003 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.759429 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.759429 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.759430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.759481 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.759678 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.759679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.759679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.759773 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.759836 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.760007 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.760076 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:36:09.760262 7751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.752774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:35751 (local address 127.0.0.1:60516)
0806 06:36:09.753292 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.753301 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.753362 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.753556 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.753557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.753809 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.753919 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.753921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.754905 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.754906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.755805 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.755807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.756307 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.756308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.756397 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.756847 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.756853 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.757388 (+ 535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.759088 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.759089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.759090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.759354 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.759544 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.759545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.759546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.759619 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.759836 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.759837 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.759906 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.759993 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.760149 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":49,"thread_start_us":97,"threads_started":1}
I20260806 06:36:09.779259 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 6774
I20260806 06:36:09.786473 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.752621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54212 (local address 127.29.142.189:41567)
0806 06:36:09.752827 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.752829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.753170 (+ 341us) server_negotiation.cc:407] Connection header received
0806 06:36:09.753252 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.753253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.753264 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.753311 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.753705 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.754515 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.755809 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.756117 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.756160 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.780616 (+ 24456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.780619 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.780620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.780629 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.785044 (+ 4415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.785547 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.785548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.785549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.785616 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.785824 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.785824 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.785825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.785982 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.786030 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.786250 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.786362 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260806 06:36:09.786854 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.752614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:41567 (local address 127.0.0.1:54212)
0806 06:36:09.753024 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.753033 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.753186 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.753370 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.753371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.753538 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.753616 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.753617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.754635 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.754636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.755697 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.755699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.756276 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.756277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.756357 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.779261 (+ 22904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.779286 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.780448 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.785176 (+ 4728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.785178 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.785179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.785446 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.785691 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.785692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.785692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.785751 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.786081 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.786082 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.786134 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.786579 (+ 445us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.786738 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":33,"thread_start_us":92,"threads_started":1}
I20260806 06:36:09.793962 7752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.752514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42322)
0806 06:36:09.752902 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.752913 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.753018 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.753195 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.753196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.753469 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.753592 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.753595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.754635 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.754636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.755697 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.755699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.756278 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.756279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.756342 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.789910 (+ 33568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.789917 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.790412 (+ 495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.792650 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.792652 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.792652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.792910 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.793149 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.793150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.793151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.793224 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.793530 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.793531 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.793610 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.793687 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.793820 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":40,"thread_start_us":102,"threads_started":1}
I20260806 06:36:09.793998 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.752763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.29.142.190:39715)
0806 06:36:09.752809 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.752811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.752973 (+ 162us) server_negotiation.cc:407] Connection header received
0806 06:36:09.753097 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.753099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.753110 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.753161 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.753702 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.754513 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.755809 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.756118 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.756161 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.790539 (+ 34378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.790542 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.790542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.790553 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.792543 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.792999 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.793001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.793001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.793074 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.793292 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.793293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.793294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.793458 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.793529 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.793738 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.793848 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:36:09.795260 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:41849
--local_ip_for_outbound_sockets=127.29.142.129
--tserver_master_addrs=127.29.142.190:39715
--webserver_port=39427
--webserver_interface=127.29.142.129
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab
--principal=oryx/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.142.190:39715,127.29.142.189:41567
--tserver_master_addrs=127.29.142.190:39715,127.29.142.189:41567,127.29.142.188:35751 with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998168, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.190@KRBTEST.COM for oryx/127.29.142.188@KRBTEST.COM
I20260806 06:36:09.805675 7495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.797020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:35751 (local address 127.0.0.1:60528)
0806 06:36:09.797075 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.797091 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.797162 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.797324 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.797325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.797499 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.797545 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.797547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.798197 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.798199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.799139 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.799142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.799703 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.799704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.799779 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.800309 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.800314 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.802111 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.804409 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.804411 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.804411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.804663 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.804909 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.804910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.804910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.804984 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.805307 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.805309 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.805362 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.805439 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.805561 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:36:09.805717 7722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.797063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60528 (local address 127.29.142.188:35751)
0806 06:36:09.797105 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.797108 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.797140 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:36:09.797237 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.797238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.797246 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.797324 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.797634 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.798110 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.799255 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.799610 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.799695 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.802239 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.802241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.802242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.802251 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.804286 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.804755 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.804757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.804757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.804828 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.805058 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.805059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.805060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.805227 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.805310 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.805484 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.805578 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:36:09.806916 6682 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.142.188:35751: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9589ecac6b124e538c642accf1810167"
member_type: NON_VOTER
last_known_addr {
host: "127.29.142.188"
port: 35751
}
attrs {
promote: true
}
}
dest_uuid: "02bc53d761e34b5385e5312657e82269"
cas_config_opid_index: 5
I20260806 06:36:09.807214 6682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: NON_VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: true } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } }
I20260806 06:36:09.807701 7497 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9589ecac6b124e538c642accf1810167 (127.29.142.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: NON_VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: true } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:09.808490 7683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:09.808784 7497 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.142.188:35751
I20260806 06:36:09.808861 7497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: NON_VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: true } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:09.808928 7497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:09.809223 7626 master.cc:537] Master@127.29.142.188:35751 shutting down...
I20260806 06:36:09.809561 7683 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 1 FOLLOWER]: Refusing update from remote peer 02bc53d761e34b5385e5312657e82269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 06:36:09.809856 7498 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:36:09.810899 6659 consensus_peers.cc:487] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 -> Peer 9589ecac6b124e538c642accf1810167 (127.29.142.188:35751): Couldn't send request to peer 9589ecac6b124e538c642accf1810167. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 06:36:09.815248 7626 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:09.815369 7626 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:09.815416 7626 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: stopping tablet replica
I20260806 06:36:09.815989 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.807982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54222 (local address 127.29.142.189:41567)
0806 06:36:09.808050 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.808052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.811834 (+ 3782us) server_negotiation.cc:407] Connection header received
0806 06:36:09.811946 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.811947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.811974 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.812014 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:09.812219 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.813023 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.814518 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.815244 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.815363 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.815451 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.815761 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.815901 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:36:09.816265 7495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.807872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:41567 (local address 127.0.0.1:54222)
0806 06:36:09.811762 (+ 3890us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.811777 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.811867 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.812098 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.812099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.812100 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:09.812147 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.812148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.813126 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.813127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.814407 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.814410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.815490 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.815491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.815606 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.815642 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.816102 (+ 460us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.816171 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3868}
I20260806 06:36:09.816766 7572 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 LEARNER]: Refusing update from remote peer 02bc53d761e34b5385e5312657e82269: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 06:36:09.817118 7498 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: NON_VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 06:36:09.817816 7572 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9589ecac6b124e538c642accf1810167 (127.29.142.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: NON_VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: true } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:09.822422 7498 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269: attempting to promote NON_VOTER 7619e678ceb947c49d8eafffa4b13272 to VOTER
I20260806 06:36:09.822742 7498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } }
I20260806 06:36:09.823472 7572 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 LEARNER]: Refusing update from remote peer 02bc53d761e34b5385e5312657e82269: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 06:36:09.823791 7498 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 06:36:09.824285 7495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.823240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:35751 (local address 127.0.0.1:60534)
0806 06:36:09.824173 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.824229 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:35751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":913}
I20260806 06:36:09.824677 7495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.824513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:35751 (local address 127.0.0.1:60536)
0806 06:36:09.824625 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.824650 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:35751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
W20260806 06:36:09.824724 6659 consensus_peers.cc:487] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 -> Peer 9589ecac6b124e538c642accf1810167 (127.29.142.188:35751): Couldn't send request to peer 9589ecac6b124e538c642accf1810167. Status: Network error: Client connection negotiation failed: client connection to 127.29.142.188:35751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 06:36:09.825090 7497 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7619e678ceb947c49d8eafffa4b13272 (127.29.142.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:09.826040 7497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:09.826112 7497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:09.826162 7497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:09.826201 7497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:09.826437 7572 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7619e678ceb947c49d8eafffa4b13272 (127.29.142.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:09.831063 7626 master.cc:552] Master@127.29.142.188:35751 shutdown complete.
I20260806 06:36:09.831133 7626 master_runner.cc:305] Clearing existing system tablet
I20260806 06:36:09.831295 7626 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:09.831342 7626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:09.831386 7610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02bc53d761e34b5385e5312657e82269. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:09.831498 7610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:36:09.831538 7610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:09.831574 7610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:36:09.831600 7610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:09.831633 7610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:36:09.832370 7626 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:09.832957 7766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.833110 7626 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:09.833182 7626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.833209 7626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/wal
uuid: "9589ecac6b124e538c642accf1810167"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "0874e19551652b2a7be2214e9f43bfc9"
server_key_iv: "d7dd70a3800ddd4ff44fbdf50cb01f1e"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.834019 7626 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:09.835678 7626 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 06:36:09.835747 7626 log.cc:1196] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 06:36:09.835973 7626 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: Deleting consensus metadata
I20260806 06:36:09.836107 7626 master_runner.cc:315] Copying system tablet from 127.29.142.190:39715
I20260806 06:36:09.836812 7626 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: tablet copy: Beginning tablet copy session from remote peer at address 127.29.142.190:39715
I20260806 06:36:09.844745 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.837369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42328 (local address 127.29.142.190:39715)
0806 06:36:09.837418 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.837421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.837434 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:36:09.837502 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.837503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.837514 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.837581 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.837998 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.838830 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.839900 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.840233 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.840318 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.841529 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.841531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.841531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.841541 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.843411 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.843845 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.843847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.843847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.843915 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.844147 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.844148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.844148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.844298 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.844372 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.844556 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.844644 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:36:09.844754 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.837174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42328)
0806 06:36:09.837337 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.837352 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.837433 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.837589 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.837590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.837813 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.837877 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.837880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.838944 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.838945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.839801 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.839804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.840315 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.840316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.840390 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.840970 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.840976 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.841423 (+ 447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.843515 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.843517 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.843517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.843768 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.843994 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.843995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.843996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.844072 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.844378 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.844380 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.844435 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.844518 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.844646 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260806 06:36:09.845537 6709 tablet_copy_service.cc:140] P 02bc53d761e34b5385e5312657e82269: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9589ecac6b124e538c642accf1810167 ({username='oryx', principal='oryx/127.29.142.188@KRBTEST.COM'} at 127.0.0.1:42328)
I20260806 06:36:09.845610 6709 tablet_copy_service.cc:161] P 02bc53d761e34b5385e5312657e82269: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9589ecac6b124e538c642accf1810167 at {username='oryx', principal='oryx/127.29.142.188@KRBTEST.COM'} at 127.0.0.1:42328: session id = 9589ecac6b124e538c642accf1810167-00000000000000000000000000000000
I20260806 06:36:09.846204 6709 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 06:36:09.846827 7626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 06:36:09.848505 7626 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: tablet copy: Starting download of 0 data blocks...
I20260806 06:36:09.848668 7626 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: tablet copy: Starting download of 1 WAL segments...
I20260806 06:36:09.849695 7626 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 06:36:09.850975 6709 tablet_copy_service.cc:342] P 02bc53d761e34b5385e5312657e82269: Request end of tablet copy session 9589ecac6b124e538c642accf1810167-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.142.188@KRBTEST.COM'} at 127.0.0.1:42328
I20260806 06:36:09.851042 6709 tablet_copy_service.cc:433] P 02bc53d761e34b5385e5312657e82269: ending tablet copy session 9589ecac6b124e538c642accf1810167-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9589ecac6b124e538c642accf1810167
I20260806 06:36:09.854512 7626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:09.855878 7775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:09.859885 7776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:09.860227 7626 server_base.cc:1034] running on GCE node
W20260806 06:36:09.862819 7781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:09.863086 7626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:09.863302 7626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:09.864456 7626 hybrid_clock.cc:648] HybridClock initialized: now 1785998169864435 us; error 46 us; skew 500 ppm
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:09.866158 7626 init.cc:377] Logged in from keytab as oryx/127.29.142.188@KRBTEST.COM (short username oryx)
I20260806 06:36:09.866483 7626 webserver.cc:460] Webserver started at http://127.29.142.188:45673/ using document root <none> and password file <none>
I20260806 06:36:09.866613 7626 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:09.866688 7626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:09.867527 7626 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:09.868196 7790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.868382 7626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:36:09.868439 7626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/wal
uuid: "9589ecac6b124e538c642accf1810167"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "0874e19551652b2a7be2214e9f43bfc9"
server_key_iv: "d7dd70a3800ddd4ff44fbdf50cb01f1e"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.868505 7626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:09.883885 7626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:09.886204 7626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:09.890426 7626 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.188:35751
I20260806 06:36:09.890475 7864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.188:35751 every 8 connection(s)
I20260806 06:36:09.890882 7626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
I20260806 06:36:09.892068 7865 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:36:09.892596 7865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: Bootstrap starting.
I20260806 06:36:09.896147 7865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:36:09.896394 7865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: Bootstrap complete.
I20260806 06:36:09.896620 7865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } }
I20260806 06:36:09.896716 7865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9589ecac6b124e538c642accf1810167, State: Initialized, Role: LEARNER
I20260806 06:36:09.896770 7865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } }
I20260806 06:36:09.896983 7865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:09.896996 7868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:09.897249 7868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:36:09.898581 7883 catalog_manager.cc:1259] Loaded cluster ID: 441134192b9a48d5b70cc90a71af07bb
I20260806 06:36:09.898723 7883 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: loading cluster ID for follower catalog manager: success
I20260806 06:36:09.900214 7883 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: acquiring CA information for follower catalog manager: success
I20260806 06:36:09.900609 7883 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 06:36:09.910872 7754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:09.911104 7754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:09.911130 7754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:09.912364 7754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:09.912415 7754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:09.912454 7754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:36:09.914038 7754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab
--principal=oryx/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:41849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=39427
--tserver_master_addrs=127.29.142.190:39715,127.29.142.189:41567,127.29.142.188:35751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:09.914304 7754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:09.914561 7754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:09.920109 7890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:09.920256 7754 server_base.cc:1034] running on GCE node
W20260806 06:36:09.920109 7889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:09.920475 7893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:09.920713 7754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:09.921046 7754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:09.922211 7754 hybrid_clock.cc:648] HybridClock initialized: now 1785998169922198 us; error 36 us; skew 500 ppm
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:09.925165 7754 init.cc:377] Logged in from keytab as oryx/127.29.142.129@KRBTEST.COM (short username oryx)
I20260806 06:36:09.925648 7754 webserver.cc:460] Webserver started at http://127.29.142.129:39427/ using document root <none> and password file <none>
I20260806 06:36:09.925910 7754 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:09.926005 7754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:09.927621 7754 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:09.928309 7903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.928522 7754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:09.928632 7754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "523ec1cf6a9e482a8fa0874e25825d9b"
format_stamp: "Formatted at 2026-08-06 06:36:08 on dist-test-slave-3kk6"
server_key: "1beda9f46319030ffbcc51fc1a21e88d"
server_key_iv: "38562232daa95d5b330ddbc75df7bd33"
server_key_version: "encryptionkey@0"
I20260806 06:36:09.929080 7754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:09.944065 7754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:09.946370 7754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:09.946568 7754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:09.946936 7754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:09.947304 7754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:09.947358 7754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.947396 7754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:09.947438 7754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:09.956046 7754 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:41849
I20260806 06:36:09.956141 8065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:41849 every 8 connection(s)
I20260806 06:36:09.956570 7754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:36:09.962694 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 7754
I20260806 06:36:09.962908 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 6964
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.189@KRBTEST.COM
Aug 06 06:36:09 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998169, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.188@KRBTEST.COM
I20260806 06:36:09.971318 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.958141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:48641 (local address 127.29.142.190:39715)
0806 06:36:09.958207 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.958209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.958920 (+ 711us) server_negotiation.cc:407] Connection header received
0806 06:36:09.959512 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.959513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.959527 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.959561 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.960548 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.961070 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.962637 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.962986 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.963030 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.967053 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.967056 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.967056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.967066 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.969356 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.969972 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.969973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.969974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.970040 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.970393 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.970394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.970395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.970556 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.970602 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.971085 (+ 483us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.971194 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:36:09.971611 8070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.957529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:45671 (local address 127.29.142.188:35751)
0806 06:36:09.957713 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.957715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.958051 (+ 336us) server_negotiation.cc:407] Connection header received
0806 06:36:09.959640 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.959642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.959656 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.959779 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.962065 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.963080 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.964629 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.964948 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.965000 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.967526 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.967527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.967527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.967536 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.970208 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.970637 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.970639 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.970640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.970735 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.970962 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.970963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.970963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.971107 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.971152 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.971362 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.971485 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":75,"threads_started":1}
I20260806 06:36:09.972193 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.957593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:50841 (local address 127.29.142.189:41567)
0806 06:36:09.957652 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:09.957655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:09.958051 (+ 396us) server_negotiation.cc:407] Connection header received
0806 06:36:09.959486 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:09.959488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:09.959506 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:09.959570 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:09.961205 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.961937 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.962955 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.963326 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.963393 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.967074 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:09.967076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:09.967076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:09.967086 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:09.969896 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.970514 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.970516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.970516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.970593 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:09.970898 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:09.970899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:09.970899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:09.971107 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:09.971189 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:09.971914 (+ 725us) server_negotiation.cc:299] Negotiation successful
0806 06:36:09.972034 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:36:09.972282 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.957471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:35751 (local address 127.29.142.129:45671)
0806 06:36:09.957930 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.957952 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.959356 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.959793 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.959794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.960211 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.960433 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.960440 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.963173 (+ 2733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.963174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.964515 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.964518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.965095 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.965096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.965182 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.965770 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.965775 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.967427 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.970297 (+ 2870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.970297 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.970298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.970539 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.970812 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.970813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.970813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.970883 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.971205 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.971206 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.971245 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.971716 (+ 471us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.972014 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":128,"thread_start_us":378,"threads_started":1}
I20260806 06:36:09.972496 8075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.958061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.129:48641)
0806 06:36:09.958828 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.958842 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.959369 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.959662 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.959663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.960169 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.960433 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.960440 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.961209 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.961210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.962522 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.962525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.963188 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.963189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.963335 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.963973 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.963986 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.966912 (+ 2926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.969469 (+ 2557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.969483 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.969487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.969855 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.970123 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.970124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.970124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.970296 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.970688 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.970698 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.970904 (+ 206us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.971608 (+ 704us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.971960 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":85,"threads_started":1}
I20260806 06:36:09.973013 8066 heartbeater.cc:347] Connected to a master server at 127.29.142.188:35751
I20260806 06:36:09.973042 8067 heartbeater.cc:347] Connected to a master server at 127.29.142.190:39715
I20260806 06:36:09.973150 8067 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:09.973150 8066 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:09.973400 8066 heartbeater.cc:511] Master 127.29.142.188:35751 requested a full tablet report, sending...
I20260806 06:36:09.973400 8067 heartbeater.cc:511] Master 127.29.142.190:39715 requested a full tablet report, sending...
I20260806 06:36:09.973824 8072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:09.957470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:41567 (local address 127.29.142.129:50841)
0806 06:36:09.957893 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:09.957952 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:09.959356 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:09.959658 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:09.959663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:09.960217 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:09.960433 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.960440 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.962032 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.962033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.962839 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:09.962843 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:09.963521 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:09.963522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:09.963633 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:09.964111 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:09.964117 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:09.966931 (+ 2814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:09.970011 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.970012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.970013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.970410 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.970733 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:09.970734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:09.970735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:09.970809 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:09.971223 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:09.971224 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:09.971274 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 06:36:09.971377 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 06:36:09.973176 (+ 1799us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":1,"thread_start_us":160,"threads_started":1}
I20260806 06:36:09.973961 6682 ts_manager.cc:194] Re-registered known tserver with Master: 523ec1cf6a9e482a8fa0874e25825d9b (127.29.142.129:41849)
I20260806 06:36:09.974056 7816 ts_manager.cc:194] Registered new tserver with Master: 523ec1cf6a9e482a8fa0874e25825d9b (127.29.142.129:41849)
I20260806 06:36:09.974575 6682 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:48641
I20260806 06:36:09.988351 8068 heartbeater.cc:347] Connected to a master server at 127.29.142.189:41567
I20260806 06:36:09.988446 8068 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:09.988583 8068 heartbeater.cc:511] Master 127.29.142.189:41567 requested a full tablet report, sending...
I20260806 06:36:09.989073 7556 ts_manager.cc:194] Registered new tserver with Master: 523ec1cf6a9e482a8fa0874e25825d9b (127.29.142.129:41849)
I20260806 06:36:09.993047 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:39021
--local_ip_for_outbound_sockets=127.29.142.130
--tserver_master_addrs=127.29.142.190:39715
--webserver_port=43341
--webserver_interface=127.29.142.130
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab
--principal=oryx/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.142.190:39715,127.29.142.189:41567
--tserver_master_addrs=127.29.142.190:39715,127.29.142.189:41567,127.29.142.188:35751 with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:10.079909 8076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:10.080169 8076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:10.080212 8076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:10.081410 8076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:10.081489 8076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:10.081539 8076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:36:10.083196 8076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab
--principal=oryx/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:39021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=43341
--tserver_master_addrs=127.29.142.190:39715,127.29.142.189:41567,127.29.142.188:35751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:10.083500 8076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:10.083765 8076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:10.089303 8085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:10.089287 8084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:10.089250 8090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:10.089596 8076 server_base.cc:1034] running on GCE node
I20260806 06:36:10.089879 8076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:10.090149 8076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:10.091325 8076 hybrid_clock.cc:648] HybridClock initialized: now 1785998170091312 us; error 43 us; skew 500 ppm
Aug 06 06:36:10 dist-test-slave-3kk6 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998170, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:10.098217 8076 init.cc:377] Logged in from keytab as oryx/127.29.142.130@KRBTEST.COM (short username oryx)
I20260806 06:36:10.098874 8076 webserver.cc:460] Webserver started at http://127.29.142.130:43341/ using document root <none> and password file <none>
I20260806 06:36:10.099156 8076 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:10.099236 8076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:10.100816 8076 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:36:10.101663 8099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:10.101886 8076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:10.101986 8076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "270b539b1e414d63952194c491296c18"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "4bc86c4ac4a752527942ca6b1f1e9d20"
server_key_iv: "ec2531af528d00db74e596e3dea7c7d1"
server_key_version: "encryptionkey@0"
I20260806 06:36:10.102456 8076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:10.120898 8076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:10.123252 8076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:10.123380 8076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:10.123605 8076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:10.123905 8076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:10.123941 8076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:10.123965 8076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:10.123977 8076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:10.130262 8076 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:39021
I20260806 06:36:10.130326 8261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:39021 every 8 connection(s)
I20260806 06:36:10.130805 8076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:36:10 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998170, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
I20260806 06:36:10.139187 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 8076
I20260806 06:36:10.139359 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 7151
Aug 06 06:36:10 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998170, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.188@KRBTEST.COM
Aug 06 06:36:10 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998170, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.189@KRBTEST.COM
I20260806 06:36:10.145149 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.132091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:54135 (local address 127.29.142.190:39715)
0806 06:36:10.132139 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.132141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.132359 (+ 218us) server_negotiation.cc:407] Connection header received
0806 06:36:10.133375 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.133377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.133393 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.133459 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:10.134292 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.134889 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.136776 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.137088 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.137152 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.139612 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:10.139613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:10.139614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:10.139623 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:10.143624 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.144118 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.144119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.144120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.144178 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.144397 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.144398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.144399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.144526 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:10.144569 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.144947 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.145043 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:36:10.145149 8070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.132153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:37101 (local address 127.29.142.188:35751)
0806 06:36:10.132226 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.132228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.132239 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:10.133456 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.133457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.133471 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.133525 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:10.134297 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.135102 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.136210 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.136520 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.136585 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.140036 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:10.140038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:10.140038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:10.140047 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:10.143335 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.143959 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.143960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.143961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.144024 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.144340 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.144340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.144341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.144487 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:10.144533 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.144947 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.145044 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:36:10.145360 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.131976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:46745 (local address 127.29.142.189:41567)
0806 06:36:10.132026 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.132029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.132040 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:10.133285 (+ 1245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.133287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.133301 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.133375 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:10.134295 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.135106 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.136357 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.136651 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.136715 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.140936 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:10.140937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:10.140937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:10.140945 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:10.143621 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.144211 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.144213 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.144214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.144300 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.144569 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.144570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.144570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.144745 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:10.144801 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.145078 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.145188 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:36:10.145825 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.132004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.130:54135)
0806 06:36:10.132302 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:10.132311 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:10.133184 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:10.133468 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:10.133469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:10.133907 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:10.134174 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.134179 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.134998 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.134999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.136267 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.136268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.137168 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.137169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.137255 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.137697 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:10.137702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:10.139487 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:10.143716 (+ 4229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:10.143717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:10.143717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:10.144029 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:10.144251 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:10.144251 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:10.144252 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:10.144307 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:10.144617 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:10.144618 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:10.144775 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 06:36:10.145302 (+ 527us) client_negotiation.cc:240] Negotiation successful
0806 06:36:10.145555 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":128,"thread_start_us":75,"threads_started":1}
I20260806 06:36:10.145846 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.131664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:35751 (local address 127.29.142.130:37101)
0806 06:36:10.132172 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:10.132183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:10.133187 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:10.133575 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:10.133576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:10.133920 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:10.134174 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.134181 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.135185 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.135186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.136106 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.136109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.136611 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.136611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.136767 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.137567 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:10.137572 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:10.139963 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:10.143444 (+ 3481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:10.143455 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:10.143458 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:10.143863 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:10.144105 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:10.144106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:10.144106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:10.144259 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:10.144585 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:10.144592 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:10.144775 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 06:36:10.145302 (+ 527us) client_negotiation.cc:240] Negotiation successful
0806 06:36:10.145605 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":244,"thread_start_us":117,"threads_started":1}
I20260806 06:36:10.147037 8267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.131664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:41567 (local address 127.29.142.130:46745)
0806 06:36:10.131878 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:10.131927 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:10.133166 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:10.133454 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:10.133458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:10.133912 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:10.134174 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.134181 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.135306 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.135306 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.136265 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.136268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.136716 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.136717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.136803 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.137544 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:10.137556 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:10.140841 (+ 3285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:10.143734 (+ 2893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:10.143735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:10.143736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:10.144099 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:10.144395 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:10.144396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:10.144396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:10.144474 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:10.144859 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:10.144860 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:10.144920 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 06:36:10.146479 (+ 1559us) client_negotiation.cc:240] Negotiation successful
0806 06:36:10.146625 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":32,"thread_start_us":117,"threads_started":1}
I20260806 06:36:10.147646 8265 heartbeater.cc:347] Connected to a master server at 127.29.142.189:41567
I20260806 06:36:10.147800 8265 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:10.148138 8265 heartbeater.cc:511] Master 127.29.142.189:41567 requested a full tablet report, sending...
I20260806 06:36:10.148851 7556 ts_manager.cc:194] Registered new tserver with Master: 270b539b1e414d63952194c491296c18 (127.29.142.130:39021)
I20260806 06:36:10.152814 8264 heartbeater.cc:347] Connected to a master server at 127.29.142.190:39715
I20260806 06:36:10.152896 8264 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:10.153010 8264 heartbeater.cc:511] Master 127.29.142.190:39715 requested a full tablet report, sending...
I20260806 06:36:10.152884 8263 heartbeater.cc:347] Connected to a master server at 127.29.142.188:35751
I20260806 06:36:10.153216 8263 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:10.153298 8263 heartbeater.cc:511] Master 127.29.142.188:35751 requested a full tablet report, sending...
I20260806 06:36:10.153326 6682 ts_manager.cc:194] Re-registered known tserver with Master: 270b539b1e414d63952194c491296c18 (127.29.142.130:39021)
I20260806 06:36:10.153651 7816 ts_manager.cc:194] Registered new tserver with Master: 270b539b1e414d63952194c491296c18 (127.29.142.130:39021)
I20260806 06:36:10.153879 6682 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:54135
I20260806 06:36:10.157302 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:34625
--local_ip_for_outbound_sockets=127.29.142.131
--tserver_master_addrs=127.29.142.190:39715
--webserver_port=43533
--webserver_interface=127.29.142.131
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab
--principal=oryx/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.142.190:39715,127.29.142.189:41567
--tserver_master_addrs=127.29.142.190:39715,127.29.142.189:41567,127.29.142.188:35751 with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:10.243434 8272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:10.243671 8272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:10.243698 8272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:10.245297 8272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:10.245355 8272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:10.245386 8272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:36:10.247052 8272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab
--principal=oryx/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:34625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=43533
--tserver_master_addrs=127.29.142.190:39715,127.29.142.189:41567,127.29.142.188:35751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:10.247331 8272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:10.247588 8272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:10.253182 8283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:10.253333 8272 server_base.cc:1034] running on GCE node
W20260806 06:36:10.253185 8281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:10.253221 8280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:10.253712 8272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:10.254005 8272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:10.255192 8272 hybrid_clock.cc:648] HybridClock initialized: now 1785998170255161 us; error 44 us; skew 500 ppm
Aug 06 06:36:10 dist-test-slave-3kk6 krb5kdc[6615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998170, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:10.258026 8272 init.cc:377] Logged in from keytab as oryx/127.29.142.131@KRBTEST.COM (short username oryx)
I20260806 06:36:10.258599 8272 webserver.cc:460] Webserver started at http://127.29.142.131:43533/ using document root <none> and password file <none>
I20260806 06:36:10.258867 8272 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:10.258958 8272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:10.260494 8272 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:10.261164 8294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:10.261351 8272 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:10.261461 8272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "4e615fbf098747529b61f02005270202"
format_stamp: "Formatted at 2026-08-06 06:36:09 on dist-test-slave-3kk6"
server_key: "2c7e11a34dfff8e4aa7df16b0aac1314"
server_key_iv: "7f8aeb069b80f7937b90a81e75798dff"
server_key_version: "encryptionkey@0"
I20260806 06:36:10.261837 8272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:10.274528 8272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:10.277138 8272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:10.277328 8272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:10.277603 8272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:10.277937 8272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:10.277990 8272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:10.278033 8272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:10.278064 8272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:10.285105 8272 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:34625
I20260806 06:36:10.285156 8459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:34625 every 8 connection(s)
I20260806 06:36:10.285573 8272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:36:10.292789 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 8272
Aug 06 06:36:10 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998170, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.189@KRBTEST.COM
Aug 06 06:36:10 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998170, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.188@KRBTEST.COM
Aug 06 06:36:10 dist-test-slave-3kk6 krb5kdc[6615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998170, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
I20260806 06:36:10.300213 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.286527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:43451 (local address 127.29.142.189:41567)
0806 06:36:10.286600 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.286603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.286917 (+ 314us) server_negotiation.cc:407] Connection header received
0806 06:36:10.288285 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.288287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.288302 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.288362 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:10.289270 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.290033 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.291441 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.291684 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.291762 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.295074 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:10.295076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:10.295076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:10.295085 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:10.297361 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.298919 (+ 1558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.298920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.298920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.298979 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.299351 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.299352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.299353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.299494 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:10.299551 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.300010 (+ 459us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.300115 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:36:10.300422 8471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.286908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:35751 (local address 127.29.142.131:48305)
0806 06:36:10.287057 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:10.287067 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:10.288166 (+ 1099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:10.288474 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:10.288475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:10.288928 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:10.289148 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.289153 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.289892 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.289893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.291288 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.291290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.291700 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.291701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.291859 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.292715 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:10.292721 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:10.295497 (+ 2776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:10.298824 (+ 3327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:10.298825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:10.298825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:10.299050 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:10.299313 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:10.299314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:10.299314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:10.299373 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:10.299666 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:10.299667 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:10.299806 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 06:36:10.299949 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 06:36:10.300148 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":8,"thread_start_us":81,"threads_started":1}
I20260806 06:36:10.300577 8470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.286532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:41567 (local address 127.29.142.131:43451)
0806 06:36:10.286802 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:10.286814 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:10.288149 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:10.288465 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:10.288470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:10.288966 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:10.289149 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.289155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.290106 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.290107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.291283 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.291287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.291774 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.291775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.291858 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.292650 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:10.292662 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:10.294948 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:10.298403 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:10.298415 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:10.298418 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:10.298824 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:10.299054 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:10.299055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:10.299055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:10.299251 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:10.299586 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:10.299595 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:10.299804 (+ 209us) client_negotiation.cc:769] Sending connection context
0806 06:36:10.300361 (+ 557us) client_negotiation.cc:240] Negotiation successful
0806 06:36:10.300476 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":41,"thread_start_us":152,"threads_started":1}
I20260806 06:36:10.301355 8070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.287177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:48305 (local address 127.29.142.188:35751)
0806 06:36:10.287233 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.287235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.287245 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:36:10.288364 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.288365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.288378 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.288433 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:10.289253 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.289769 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.291392 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.291611 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.291682 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.295590 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:10.295591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:10.295592 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:10.295600 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:10.298249 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.299175 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.299176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.299176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.299234 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.299447 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.299448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.299448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.299585 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:10.299642 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.301136 (+ 1494us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.301236 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:36:10.301483 8462 heartbeater.cc:347] Connected to a master server at 127.29.142.189:41567
I20260806 06:36:10.301599 8462 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:10.301855 8462 heartbeater.cc:511] Master 127.29.142.189:41567 requested a full tablet report, sending...
I20260806 06:36:10.302328 8464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.286532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.131:49247)
0806 06:36:10.286744 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:10.286791 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:10.288165 (+ 1374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:10.288467 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:10.288470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:10.288974 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:10.289149 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.289155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.290729 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.290730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.291291 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.291293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.292269 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.292269 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.292323 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.292950 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:10.292955 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:10.296582 (+ 3627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:10.301073 (+ 4491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:10.301074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:10.301075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:10.301307 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:10.301586 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:10.301586 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:10.301587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:10.301650 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:10.301964 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:10.301965 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:10.302009 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 06:36:10.302063 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:36:10.302189 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":25,"thread_start_us":133,"threads_started":1}
I20260806 06:36:10.302531 7556 ts_manager.cc:194] Registered new tserver with Master: 4e615fbf098747529b61f02005270202 (127.29.142.131:34625)
I20260806 06:36:10.302712 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.286682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:49247 (local address 127.29.142.190:39715)
0806 06:36:10.286732 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.286734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.286945 (+ 211us) server_negotiation.cc:407] Connection header received
0806 06:36:10.288290 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.288292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.288306 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.288339 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:10.289270 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.290031 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.291838 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.292143 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.292207 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.296692 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:10.296694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:10.296695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:10.296703 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:10.300989 (+ 4286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.301404 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.301406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.301406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.301503 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.301723 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.301724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.301724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.301865 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:10.301933 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.302482 (+ 549us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.302573 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:36:10.303323 8460 heartbeater.cc:347] Connected to a master server at 127.29.142.188:35751
I20260806 06:36:10.303375 8460 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:10.303426 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.294795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54236 (local address 127.29.142.189:41567)
0806 06:36:10.294906 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.294908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.294917 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:36:10.294943 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.294944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.294958 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.295951 (+ 993us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:10.295995 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.296814 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.297898 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.298208 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.298526 (+ 318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.299747 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:10.299748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:10.299748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:10.299755 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:10.301865 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.302310 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.302311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.302311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.302374 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.302591 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.302592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.302592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.302769 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:10.302880 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.303222 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.303329 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":64,"threads_started":1}
I20260806 06:36:10.303448 8460 heartbeater.cc:511] Master 127.29.142.188:35751 requested a full tablet report, sending...
I20260806 06:36:10.303920 7816 ts_manager.cc:194] Registered new tserver with Master: 4e615fbf098747529b61f02005270202 (127.29.142.131:34625)
I20260806 06:36:10.304226 8461 heartbeater.cc:347] Connected to a master server at 127.29.142.190:39715
I20260806 06:36:10.304288 8461 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:10.304363 8461 heartbeater.cc:511] Master 127.29.142.190:39715 requested a full tablet report, sending...
I20260806 06:36:10.304630 6682 ts_manager.cc:194] Re-registered known tserver with Master: 4e615fbf098747529b61f02005270202 (127.29.142.131:34625)
I20260806 06:36:10.305152 6682 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:49247
I20260806 06:36:10.307168 8484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.294221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60548 (local address 127.29.142.188:35751)
0806 06:36:10.294400 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.294402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.296094 (+ 1692us) server_negotiation.cc:407] Connection header received
0806 06:36:10.296458 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.296459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.296474 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.296516 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:10.297570 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.298340 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.299503 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.299826 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.299870 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.303865 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:10.303866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:10.303867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:10.303874 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:10.305737 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.306197 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.306198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.306199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.306263 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.306512 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.306513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.306513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.306734 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:10.306828 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.306973 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.307043 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":157,"thread_start_us":131,"threads_started":1}
I20260806 06:36:10.309931 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.294920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42340 (local address 127.29.142.190:39715)
0806 06:36:10.298692 (+ 3772us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.298694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.298701 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:36:10.298725 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.298726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.298739 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.302982 (+ 4243us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:10.303669 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.304777 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.305862 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.306113 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.306174 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.307797 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:10.307799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:10.307799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:10.307806 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:10.308986 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.309301 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.309302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.309302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.309351 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.309465 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.309465 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.309466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.309553 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:10.309621 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.309715 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.309799 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3673,"thread_start_us":177,"threads_started":1}
I20260806 06:36:10.311611 6682 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:36:10.313169 6682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:36:10.321759 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.317812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40580 (local address 127.29.142.129:41849)
0806 06:36:10.318039 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.318045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.318059 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:36:10.318104 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.318107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.318125 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.318266 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:10.318405 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.319586 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.320841 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.321398 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.321518 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.321586 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.321626 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.321666 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":103,"threads_started":1}
I20260806 06:36:10.321738 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.317469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:41849 (local address 127.0.0.1:40580)
0806 06:36:10.317829 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:10.317840 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:10.318031 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:10.318267 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:10.318268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:10.318270 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:10.318320 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.318321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.319671 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.319672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.320767 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.320769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.321491 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.321493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.321556 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.321560 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:36:10.321600 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 06:36:10.321652 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":182,"threads_started":1}
I20260806 06:36:10.321873 7495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.317416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:34625 (local address 127.0.0.1:52784)
0806 06:36:10.317732 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:10.317741 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:10.317769 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:10.318755 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:10.318756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:10.318758 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:10.318797 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.318798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.319963 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.319965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.320681 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.320683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.321610 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.321611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.321692 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.321699 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:36:10.321731 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 06:36:10.321788 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":28}
I20260806 06:36:10.322057 7762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.317416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:39021 (local address 127.0.0.1:58102)
0806 06:36:10.317576 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:10.317591 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:10.317665 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:10.318383 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:10.318384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:10.318386 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:10.318420 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.318421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.319470 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.319472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.320675 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.320678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.321610 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.321611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.321692 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.321699 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:36:10.321731 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 06:36:10.321788 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":42}
I20260806 06:36:10.322242 8496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.317568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58102 (local address 127.29.142.130:39021)
0806 06:36:10.318173 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.318177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.318188 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:10.318240 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.318243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.318260 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.318368 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:10.318486 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.319346 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.320778 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.321514 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.321858 (+ 344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.321928 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.321959 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.322012 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":114,"threads_started":1}
I20260806 06:36:10.322777 7975 tablet_service.cc:1467] Processing CreateTablet for tablet 5d3cd11b9bd543398ceecc83acd5c325 (DEFAULT_TABLE table=test-table [id=26041d12a37d430ebf903d51aad43e35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:10.323020 8160 tablet_service.cc:1467] Processing CreateTablet for tablet 5d3cd11b9bd543398ceecc83acd5c325 (DEFAULT_TABLE table=test-table [id=26041d12a37d430ebf903d51aad43e35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:10.323203 7975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d3cd11b9bd543398ceecc83acd5c325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:10.323454 8160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d3cd11b9bd543398ceecc83acd5c325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:10.323477 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.317524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52784 (local address 127.29.142.131:34625)
0806 06:36:10.318522 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.318527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.318536 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:36:10.318578 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.318580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.318596 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.318742 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:10.318857 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.319865 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.320803 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.321500 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.323244 (+ 1744us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.323309 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.323340 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.323389 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":112,"threads_started":1}
I20260806 06:36:10.324173 8359 tablet_service.cc:1467] Processing CreateTablet for tablet 5d3cd11b9bd543398ceecc83acd5c325 (DEFAULT_TABLE table=test-table [id=26041d12a37d430ebf903d51aad43e35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:10.324570 8359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d3cd11b9bd543398ceecc83acd5c325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:10.326704 8499 tablet_bootstrap.cc:492] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b: Bootstrap starting.
I20260806 06:36:10.327441 8500 tablet_bootstrap.cc:492] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18: Bootstrap starting.
I20260806 06:36:10.327790 8499 tablet_bootstrap.cc:654] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:10.328284 8500 tablet_bootstrap.cc:654] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:10.328311 8501 tablet_bootstrap.cc:492] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202: Bootstrap starting.
I20260806 06:36:10.328284 8499 log.cc:824] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:10.328771 8500 log.cc:824] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:10.329103 8501 tablet_bootstrap.cc:654] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:10.329514 8501 log.cc:824] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:10.329659 8500 tablet_bootstrap.cc:492] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18: No bootstrap required, opened a new log
I20260806 06:36:10.329757 8500 ts_tablet_manager.cc:1397] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:36:10.329957 8499 tablet_bootstrap.cc:492] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b: No bootstrap required, opened a new log
I20260806 06:36:10.330039 8499 ts_tablet_manager.cc:1397] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:36:10.330363 8501 tablet_bootstrap.cc:492] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202: No bootstrap required, opened a new log
I20260806 06:36:10.330440 8501 ts_tablet_manager.cc:1397] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:36:10.332362 8500 raft_consensus.cc:348] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:10.332551 8500 raft_consensus.cc:374] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:10.332602 8500 raft_consensus.cc:729] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270b539b1e414d63952194c491296c18, State: Initialized, Role: FOLLOWER
I20260806 06:36:10.332708 8499 raft_consensus.cc:348] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:10.332789 8500 consensus_queue.cc:260] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:10.332875 8499 raft_consensus.cc:374] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:10.332911 8499 raft_consensus.cc:729] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 523ec1cf6a9e482a8fa0874e25825d9b, State: Initialized, Role: FOLLOWER
I20260806 06:36:10.332873 8501 raft_consensus.cc:348] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:10.333026 8501 raft_consensus.cc:374] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:10.333060 8501 raft_consensus.cc:729] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e615fbf098747529b61f02005270202, State: Initialized, Role: FOLLOWER
I20260806 06:36:10.333047 8499 consensus_queue.cc:260] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:10.333108 8500 ts_tablet_manager.cc:1428] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:36:10.333204 8501 consensus_queue.cc:260] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:10.333434 8067 heartbeater.cc:503] Master 127.29.142.190:39715 was elected leader, sending a full tablet report...
I20260806 06:36:10.333491 8264 heartbeater.cc:503] Master 127.29.142.190:39715 was elected leader, sending a full tablet report...
I20260806 06:36:10.334252 8499 ts_tablet_manager.cc:1428] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 06:36:10.335250 8501 ts_tablet_manager.cc:1428] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260806 06:36:10.335290 8461 heartbeater.cc:503] Master 127.29.142.190:39715 was elected leader, sending a full tablet report...
I20260806 06:36:10.338577 8484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.335957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60558 (local address 127.29.142.188:35751)
0806 06:36:10.336004 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.336007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.336237 (+ 230us) server_negotiation.cc:407] Connection header received
0806 06:36:10.336262 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.336263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.336287 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.336351 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:10.336512 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.337046 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.337926 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.338324 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.338388 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.338437 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.338498 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.338526 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:36:10.338584 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.335852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:35751 (local address 127.0.0.1:60558)
0806 06:36:10.336120 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:10.336134 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:10.336181 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:10.336354 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:10.336355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:10.336357 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:10.336403 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.336405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.337116 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.337117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.337837 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.337839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.338396 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.338397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.338442 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.338446 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:36:10.338492 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:36:10.338531 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247}
I20260806 06:36:10.339013 7831 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260806 06:36:10.340451 7868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02bc53d761e34b5385e5312657e82269. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:10.340552 7868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: This master's current role is: LEARNER
I20260806 06:36:10.354919 8509 raft_consensus.cc:482] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:36:10.355044 8509 raft_consensus.cc:504] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:10.355515 8509 leader_election.cc:288] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 270b539b1e414d63952194c491296c18 (127.29.142.130:39021), 523ec1cf6a9e482a8fa0874e25825d9b (127.29.142.129:41849)
I20260806 06:36:10.358508 8496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.355876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:48519 (local address 127.29.142.130:39021)
0806 06:36:10.355922 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.355924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.355932 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:36:10.355963 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.355964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.355974 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.356034 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:10.356268 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.356738 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.357742 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.358135 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.358193 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.358200 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.358386 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.358430 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:36:10.358522 8470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.355730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:41849 (local address 127.29.142.131:44029)
0806 06:36:10.355891 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:10.355903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:10.355960 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:10.356107 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:10.356107 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:10.356108 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:10.356179 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.356180 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.356853 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.356854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.357647 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.357649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.358258 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.358259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.358327 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.358329 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:36:10.358402 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:36:10.358439 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":133}
I20260806 06:36:10.358534 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.355809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:44029 (local address 127.29.142.129:41849)
0806 06:36:10.355881 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.355883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.355954 (+ 71us) server_negotiation.cc:407] Connection header received
0806 06:36:10.356032 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.356033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.356043 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.356097 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:10.356246 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.356751 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.357744 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.358152 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.358242 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.358250 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.358421 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.358470 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:36:10.358533 8464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.355726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:39021 (local address 127.29.142.131:48519)
0806 06:36:10.355789 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:10.355804 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:10.355855 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:10.356048 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:10.356049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:10.356051 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:10.356175 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.356177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.356824 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.356824 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.357649 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:10.357652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.358221 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.358222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:10.358305 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.358308 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:36:10.358363 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:36:10.358417 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":36}
I20260806 06:36:10.358975 8003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d3cd11b9bd543398ceecc83acd5c325" candidate_uuid: "4e615fbf098747529b61f02005270202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" is_pre_election: true
I20260806 06:36:10.359148 8003 raft_consensus.cc:2454] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e615fbf098747529b61f02005270202 in term 0.
I20260806 06:36:10.359162 8191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d3cd11b9bd543398ceecc83acd5c325" candidate_uuid: "4e615fbf098747529b61f02005270202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270b539b1e414d63952194c491296c18" is_pre_election: true
I20260806 06:36:10.359339 8191 raft_consensus.cc:2454] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e615fbf098747529b61f02005270202 in term 0.
I20260806 06:36:10.359515 8297 leader_election.cc:302] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e615fbf098747529b61f02005270202, 523ec1cf6a9e482a8fa0874e25825d9b; no voters:
I20260806 06:36:10.359697 8509 raft_consensus.cc:2781] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:36:10.359761 8509 raft_consensus.cc:482] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:36:10.359788 8509 raft_consensus.cc:3037] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:10.360579 8509 raft_consensus.cc:504] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:10.360733 8509 leader_election.cc:288] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [CANDIDATE]: Term 1 election: Requested vote from peers 270b539b1e414d63952194c491296c18 (127.29.142.130:39021), 523ec1cf6a9e482a8fa0874e25825d9b (127.29.142.129:41849)
I20260806 06:36:10.360899 8191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d3cd11b9bd543398ceecc83acd5c325" candidate_uuid: "4e615fbf098747529b61f02005270202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270b539b1e414d63952194c491296c18"
I20260806 06:36:10.360906 8003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d3cd11b9bd543398ceecc83acd5c325" candidate_uuid: "4e615fbf098747529b61f02005270202" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "523ec1cf6a9e482a8fa0874e25825d9b"
I20260806 06:36:10.360978 8191 raft_consensus.cc:3037] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:10.361001 8003 raft_consensus.cc:3037] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:10.361747 8191 raft_consensus.cc:2454] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e615fbf098747529b61f02005270202 in term 1.
I20260806 06:36:10.361846 8003 raft_consensus.cc:2454] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e615fbf098747529b61f02005270202 in term 1.
I20260806 06:36:10.361937 8299 leader_election.cc:302] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 270b539b1e414d63952194c491296c18, 4e615fbf098747529b61f02005270202; no voters:
I20260806 06:36:10.362046 8509 raft_consensus.cc:2781] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:10.362268 8509 raft_consensus.cc:686] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 1 LEADER]: Becoming Leader. State: Replica: 4e615fbf098747529b61f02005270202, State: Running, Role: LEADER
I20260806 06:36:10.362393 8509 consensus_queue.cc:237] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:10.363394 6681 catalog_manager.cc:5495] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e615fbf098747529b61f02005270202 (127.29.142.131). New cstate: current_term: 1 leader_uuid: "4e615fbf098747529b61f02005270202" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } health_report { overall_health: UNKNOWN } } }
I20260806 06:36:10.364042 7498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: NON_VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: true } } }
I20260806 06:36:10.364146 7498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:10.364915 7498 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269: attempting to promote NON_VOTER 9589ecac6b124e538c642accf1810167 to VOTER
I20260806 06:36:10.365073 7498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } }
I20260806 06:36:10.365698 7831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 1 LEARNER]: Refusing update from remote peer 02bc53d761e34b5385e5312657e82269: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 06:36:10.365779 7572 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 FOLLOWER]: Refusing update from remote peer 02bc53d761e34b5385e5312657e82269: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 06:36:10.365933 8520 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260806 06:36:10.366020 7497 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260806 06:36:10.366683 8520 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 9589ecac6b124e538c642accf1810167 (127.29.142.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } } }
I20260806 06:36:10.366801 7572 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 9589ecac6b124e538c642accf1810167 (127.29.142.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } } }
I20260806 06:36:10.366847 7831 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 9589ecac6b124e538c642accf1810167 (127.29.142.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } } }
I20260806 06:36:10.367638 8520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } } }
I20260806 06:36:10.367693 7498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } } }
I20260806 06:36:10.367767 7498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:10.367730 8520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:10.367900 8520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } } }
I20260806 06:36:10.367971 8520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:10.367986 7868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } } }
I20260806 06:36:10.368088 7868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:36:10.368256 8522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "02bc53d761e34b5385e5312657e82269" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } } }
I20260806 06:36:10.368364 8522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:36:10.380887 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.378320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52796 (local address 127.29.142.131:34625)
0806 06:36:10.378374 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.378376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.378384 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:36:10.378534 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.378535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.378553 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.378613 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:36:10.378805 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.379295 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.380102 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.380325 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.380384 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.380538 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:36:10.380645 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:36:10.380702 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.380736 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.380753 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260806 06:36:10.381630 8266 tablet.cc:2369] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:36:10.383141 8191 raft_consensus.cc:1261] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18 [term 1 FOLLOWER]: Refusing update from remote peer 4e615fbf098747529b61f02005270202: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:10.383169 8003 raft_consensus.cc:1261] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b [term 1 FOLLOWER]: Refusing update from remote peer 4e615fbf098747529b61f02005270202: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:10.383533 8509 consensus_queue.cc:1035] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [LEADER]: Connected to new peer: Peer: permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:10.383708 8516 consensus_queue.cc:1035] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [LEADER]: Connected to new peer: Peer: permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:10.388077 8527 mvcc.cc:205] Tried to move back new op lower bound from 7315448505886113792 to 7315448505805164544. Current Snapshot: MvccSnapshot[applied={T|T < 7315448505886113792}]
I20260806 06:36:10.389524 6682 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:10.389637 6682 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:10.391461 6682 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02bc53d761e34b5385e5312657e82269: Sending DeleteTablet for 3 replicas of tablet 5d3cd11b9bd543398ceecc83acd5c325
I20260806 06:36:10.391855 8160 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d3cd11b9bd543398ceecc83acd5c325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:10 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:58102
I20260806 06:36:10.391875 8359 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d3cd11b9bd543398ceecc83acd5c325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:10 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:52784
I20260806 06:36:10.391907 7975 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d3cd11b9bd543398ceecc83acd5c325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:10 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:40580
I20260806 06:36:10.392198 8540 tablet_replica.cc:330] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18: stopping tablet replica
I20260806 06:36:10.392222 8358 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d3cd11b9bd543398ceecc83acd5c325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:10 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:52784
I20260806 06:36:10.392444 8535 tablet_replica.cc:330] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b: stopping tablet replica
I20260806 06:36:10.392571 8535 raft_consensus.cc:2229] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:10.392694 8535 raft_consensus.cc:2258] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:10.392910 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 6633
I20260806 06:36:10.400696 8535 ts_tablet_manager.cc:1905] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:10.402191 8540 raft_consensus.cc:2229] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:10.402494 8540 raft_consensus.cc:2258] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:10.402992 8535 ts_tablet_manager.cc:1918] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:10.403086 8535 log.cc:1196] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-0/wal/wals/5d3cd11b9bd543398ceecc83acd5c325
I20260806 06:36:10.403414 8535 ts_tablet_manager.cc:1939] T 5d3cd11b9bd543398ceecc83acd5c325 P 523ec1cf6a9e482a8fa0874e25825d9b: Deleting consensus metadata
I20260806 06:36:10.403723 8540 ts_tablet_manager.cc:1905] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:10.406004 8540 ts_tablet_manager.cc:1918] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:10.406095 8540 log.cc:1196] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/5d3cd11b9bd543398ceecc83acd5c325
I20260806 06:36:10.406610 8540 ts_tablet_manager.cc:1939] T 5d3cd11b9bd543398ceecc83acd5c325 P 270b539b1e414d63952194c491296c18: Deleting consensus metadata
I20260806 06:36:10.407028 8542 tablet_replica.cc:330] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202: stopping tablet replica
I20260806 06:36:10.407172 8542 raft_consensus.cc:2229] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:36:10.407316 8542 raft_consensus.cc:2258] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 06:36:10.409149 8104 connection.cc:535] server connection from 127.0.0.1:58102 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 06:36:10.409148 7908 connection.cc:535] server connection from 127.0.0.1:40580 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:36:10.415045 8542 ts_tablet_manager.cc:1905] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:10.415606 8484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.392814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60574 (local address 127.29.142.188:35751)
0806 06:36:10.392897 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.392899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.402891 (+ 9992us) server_negotiation.cc:407] Connection header received
0806 06:36:10.406551 (+ 3660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.406553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.406568 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.406681 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:10.407773 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.408631 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.409992 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.410315 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.410398 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.412212 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:10.412213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:10.412213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:10.412221 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:10.414124 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.414557 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.414559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.414559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.414626 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.414899 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.414900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.414901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.415042 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:10.415140 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.415417 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.415507 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 06:36:10.416736 8542 ts_tablet_manager.cc:1918] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:10.416843 8542 log.cc:1196] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/5d3cd11b9bd543398ceecc83acd5c325
I20260806 06:36:10.417156 8542 ts_tablet_manager.cc:1939] T 5d3cd11b9bd543398ceecc83acd5c325 P 4e615fbf098747529b61f02005270202: Deleting consensus metadata
W20260806 06:36:10.417550 8295 connection.cc:441] server connection from 127.0.0.1:52784 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52784 (request call id 1) could send its response
W20260806 06:36:10.417752 8295 connection.cc:441] server connection from 127.0.0.1:52784 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52784 (request call id 2) could send its response
I20260806 06:36:10.419857 8070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.411347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60582 (local address 127.29.142.188:35751)
0806 06:36:10.411396 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.411398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.411404 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:36:10.411427 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.411428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.411438 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.411489 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:10.412214 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.412857 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.414263 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.414571 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.414604 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.416508 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:10.416510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:10.416510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:10.416517 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:10.418548 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.418996 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.418997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.418998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.419064 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.419287 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.419288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.419288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.419432 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:10.419533 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.419663 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.419748 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:36:10.423986 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:10.411535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.29.142.189:41567)
0806 06:36:10.411590 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:10.411592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:10.416752 (+ 5160us) server_negotiation.cc:407] Connection header received
0806 06:36:10.416813 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:10.416814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:10.416833 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:10.416886 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:10.417632 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.418462 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.419442 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:10.419772 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:10.419828 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:10.421321 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:10.421323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:10.421324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:10.421331 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:10.422794 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.423174 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.423175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.423176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.423233 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:10.423453 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:10.423454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:10.423455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:10.423575 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:10.423686 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:10.423790 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:36:10.423879 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:36:11.388726 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.388375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.129:56109)
0806 06:36:11.388564 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.388668 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":82,"threads_started":1}
W20260806 06:36:11.389007 8067 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:39715 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
I20260806 06:36:11.390266 8560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.390009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.130:35301)
0806 06:36:11.390138 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.390212 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
W20260806 06:36:11.390530 8264 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:39715 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
I20260806 06:36:11.392716 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.392495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.131:44819)
0806 06:36:11.392604 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.392676 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
W20260806 06:36:11.392935 8461 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:39715 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
I20260806 06:36:11.892342 8562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 02bc53d761e34b5385e5312657e82269)
I20260806 06:36:11.892540 8562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } }
I20260806 06:36:11.892611 8563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 02bc53d761e34b5385e5312657e82269)
I20260806 06:36:11.892751 8563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } }
I20260806 06:36:11.892922 8562 leader_election.cc:288] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02bc53d761e34b5385e5312657e82269 (127.29.142.190:39715), 9589ecac6b124e538c642accf1810167 (127.29.142.188:35751)
I20260806 06:36:11.893281 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.893090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42778)
0806 06:36:11.893190 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.893251 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":49,"threads_started":1}
W20260806 06:36:11.893424 7527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
I20260806 06:36:11.893941 8565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.893752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42790)
0806 06:36:11.893855 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.893904 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260806 06:36:11.894029 7527 leader_election.cc:334] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 02bc53d761e34b5385e5312657e82269 (127.29.142.190:39715): Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
I20260806 06:36:11.894136 8563 leader_election.cc:288] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02bc53d761e34b5385e5312657e82269 (127.29.142.190:39715), 7619e678ceb947c49d8eafffa4b13272 (127.29.142.189:41567)
I20260806 06:36:11.894230 8569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.894028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42804)
0806 06:36:11.894140 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.894198 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":49,"threads_started":1}
W20260806 06:36:11.894351 7794 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
I20260806 06:36:11.894847 8569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.894755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42818)
0806 06:36:11.894794 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.894817 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:36:11.894936 7794 leader_election.cc:334] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 02bc53d761e34b5385e5312657e82269 (127.29.142.190:39715): Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
I20260806 06:36:11.896940 8566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.893391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.29.142.188:35751)
0806 06:36:11.893516 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:11.893518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:11.893529 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:11.893561 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:11.893562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:11.893575 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:11.893655 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:11.893833 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.894874 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.896050 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.896564 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.896648 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:11.896658 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:11.896833 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 06:36:11.896874 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":47,"threads_started":1}
I20260806 06:36:11.896942 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.893247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:35751 (local address 127.0.0.1:60596)
0806 06:36:11.893368 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.893379 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:11.893456 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:11.893683 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:11.893684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:11.893688 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:11.893751 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:11.893753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.894977 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.894978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:11.895967 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:11.895969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.896647 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.896648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:11.896724 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:11.896729 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:36:11.896824 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 06:36:11.896871 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260806 06:36:11.897222 7831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7619e678ceb947c49d8eafffa4b13272" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9589ecac6b124e538c642accf1810167" is_pre_election: true
I20260806 06:36:11.897364 7831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7619e678ceb947c49d8eafffa4b13272 in term 1.
I20260806 06:36:11.897532 7527 leader_election.cc:302] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7619e678ceb947c49d8eafffa4b13272, 9589ecac6b124e538c642accf1810167; no voters: 02bc53d761e34b5385e5312657e82269
I20260806 06:36:11.897732 8562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 06:36:11.897774 8562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 02bc53d761e34b5385e5312657e82269)
I20260806 06:36:11.897794 8562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:36:11.897967 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.894181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54268 (local address 127.29.142.189:41567)
0806 06:36:11.894458 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:11.894460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:11.894471 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:11.894497 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:11.894498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:11.894509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:11.894585 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:11.894777 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.895477 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.896454 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.896952 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.897039 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:11.897050 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:11.897835 (+ 785us) server_negotiation.cc:299] Negotiation successful
0806 06:36:11.897894 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":108,"threads_started":1}
I20260806 06:36:11.898136 8568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.894041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:41567 (local address 127.0.0.1:54268)
0806 06:36:11.894339 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.894350 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:11.894403 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:11.894632 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:11.894633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:11.894635 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:11.894711 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:11.894713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.895568 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.895569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:11.896346 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:11.896373 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.897551 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.897551 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:11.897637 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:11.897642 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:36:11.897734 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 06:36:11.897781 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":1,"thread_start_us":60,"threads_started":1}
I20260806 06:36:11.898381 7572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9589ecac6b124e538c642accf1810167" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7619e678ceb947c49d8eafffa4b13272" is_pre_election: true
I20260806 06:36:11.898958 8562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } }
I20260806 06:36:11.899107 7572 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9589ecac6b124e538c642accf1810167 in current term 2: Already voted for candidate 7619e678ceb947c49d8eafffa4b13272 in this term.
I20260806 06:36:11.899115 8562 leader_election.cc:288] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [CANDIDATE]: Term 2 election: Requested vote from peers 02bc53d761e34b5385e5312657e82269 (127.29.142.190:39715), 9589ecac6b124e538c642accf1810167 (127.29.142.188:35751)
I20260806 06:36:11.899297 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.899210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42830)
0806 06:36:11.899244 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.899272 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 06:36:11.899314 7792 leader_election.cc:302] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9589ecac6b124e538c642accf1810167; no voters: 02bc53d761e34b5385e5312657e82269, 7619e678ceb947c49d8eafffa4b13272
I20260806 06:36:11.899380 7831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7619e678ceb947c49d8eafffa4b13272" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9589ecac6b124e538c642accf1810167"
I20260806 06:36:11.899448 7831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:36:11.899684 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.899559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42834)
0806 06:36:11.899637 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.899662 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 06:36:11.899762 7527 leader_election.cc:334] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 02bc53d761e34b5385e5312657e82269 (127.29.142.190:39715): Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
I20260806 06:36:11.900269 7831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7619e678ceb947c49d8eafffa4b13272 in term 2.
I20260806 06:36:11.900390 8563 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:36:11.900523 7527 leader_election.cc:302] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7619e678ceb947c49d8eafffa4b13272, 9589ecac6b124e538c642accf1810167; no voters: 02bc53d761e34b5385e5312657e82269
I20260806 06:36:11.900635 8562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:36:11.900806 8562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [term 2 LEADER]: Becoming Leader. State: Replica: 7619e678ceb947c49d8eafffa4b13272, State: Running, Role: LEADER
I20260806 06:36:11.900899 8562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } }
I20260806 06:36:11.901237 8575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7619e678ceb947c49d8eafffa4b13272. Latest consensus state: current_term: 2 leader_uuid: "7619e678ceb947c49d8eafffa4b13272" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } } }
I20260806 06:36:11.901336 8575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:11.901539 8578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:11.902062 8578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:11.902170 8578 catalog_manager.cc:1259] Loaded cluster ID: 441134192b9a48d5b70cc90a71af07bb
I20260806 06:36:11.902210 8578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:36:11.902406 8578 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:11.902496 8578 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: Loaded TSK: 0
I20260806 06:36:11.902643 8578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:36:11.958745 7556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:36:11.960544 7556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:36:11.961727 7831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [term 2 FOLLOWER]: Refusing update from remote peer 7619e678ceb947c49d8eafffa4b13272: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 06:36:11.961761 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.961612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42870)
0806 06:36:11.961672 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.961720 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:36:11.962116 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.962007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.0.0.1:42886)
0806 06:36:11.962056 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.962088 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:36:11.962111 8562 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 06:36:11.962241 7527 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 -> Peer 02bc53d761e34b5385e5312657e82269 (127.29.142.190:39715): Couldn't send request to peer 02bc53d761e34b5385e5312657e82269. Status: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 06:36:11.962903 8563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7619e678ceb947c49d8eafffa4b13272. Latest consensus state: current_term: 2 leader_uuid: "7619e678ceb947c49d8eafffa4b13272" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } } }
I20260806 06:36:11.963021 8563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:36:11.963577 8562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "7619e678ceb947c49d8eafffa4b13272" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } } }
I20260806 06:36:11.963685 8562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:11.963637 8563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "7619e678ceb947c49d8eafffa4b13272" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "02bc53d761e34b5385e5312657e82269" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39715 } } peers { permanent_uuid: "7619e678ceb947c49d8eafffa4b13272" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 41567 } attrs { promote: false } } peers { permanent_uuid: "9589ecac6b124e538c642accf1810167" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 35751 } attrs { promote: false } } }
I20260806 06:36:11.963737 8563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9589ecac6b124e538c642accf1810167 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:36:11.969233 8593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.965947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52812 (local address 127.29.142.131:34625)
0806 06:36:11.966280 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:11.966282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:11.966290 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:36:11.966319 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:11.966319 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:11.966341 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:11.966410 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:11.966529 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.967262 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.968292 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.968911 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.968996 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:11.969007 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:11.969101 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 06:36:11.969141 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":55,"threads_started":1}
I20260806 06:36:11.969249 8594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.965663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:34625 (local address 127.0.0.1:52812)
0806 06:36:11.966152 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.966163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:11.966212 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:11.966424 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:11.966425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:11.966428 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:11.966462 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:11.966463 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.967365 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.967366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:11.968191 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:11.968193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.968999 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.969000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:11.969045 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:11.969050 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:36:11.969095 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:36:11.969131 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":358,"thread_start_us":80,"threads_started":1}
I20260806 06:36:11.969424 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.965603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:39021 (local address 127.0.0.1:58110)
0806 06:36:11.965658 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.965671 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:11.965716 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:11.966105 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:11.966106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:11.966113 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:11.966254 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:11.966255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.967328 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.967329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:11.968443 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:11.968446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.969210 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.969211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:11.969270 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:11.969275 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:36:11.969330 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:36:11.969373 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":126}
I20260806 06:36:11.969642 8591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.965730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58110 (local address 127.29.142.130:39021)
0806 06:36:11.965884 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:11.965886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:11.965895 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:36:11.965995 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:11.965995 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:11.966008 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:11.966145 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:11.966395 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.967233 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.968543 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.969129 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.969204 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:11.969212 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:11.969506 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 06:36:11.969545 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":109,"threads_started":1}
I20260806 06:36:11.969894 8160 tablet_service.cc:1467] Processing CreateTablet for tablet 66d2b8bbec434b8e81f64d40037b38d6 (DEFAULT_TABLE table=test-table [id=f203923a28264adda9acb2298e9f12e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:11.970033 8358 tablet_service.cc:1467] Processing CreateTablet for tablet 66d2b8bbec434b8e81f64d40037b38d6 (DEFAULT_TABLE table=test-table [id=f203923a28264adda9acb2298e9f12e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:11.970064 8160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d2b8bbec434b8e81f64d40037b38d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:11.970181 8358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d2b8bbec434b8e81f64d40037b38d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:11.971093 8565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.965663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:41849 (local address 127.0.0.1:40594)
0806 06:36:11.965710 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.965720 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:11.965770 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:11.966105 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:11.966106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:11.966109 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:11.966192 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:11.966194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.967186 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.967187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:11.968445 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:11.968447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.970806 (+ 2359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.970807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:11.970889 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:11.970894 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:36:11.970961 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:36:11.971014 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":31}
I20260806 06:36:11.971105 8592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.965805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40594 (local address 127.29.142.129:41849)
0806 06:36:11.965942 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:11.965944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:11.965953 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:36:11.965985 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:11.965986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:11.965997 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:11.966342 (+ 345us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:11.966405 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.967094 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.969258 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:11.970700 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:11.970803 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:11.970813 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:11.970980 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 06:36:11.971032 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260806 06:36:11.971406 7975 tablet_service.cc:1467] Processing CreateTablet for tablet 66d2b8bbec434b8e81f64d40037b38d6 (DEFAULT_TABLE table=test-table [id=f203923a28264adda9acb2298e9f12e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:11.971531 7975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66d2b8bbec434b8e81f64d40037b38d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:11.972083 8600 tablet_bootstrap.cc:492] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202: Bootstrap starting.
I20260806 06:36:11.972520 8600 tablet_bootstrap.cc:654] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:11.973042 8606 tablet_bootstrap.cc:492] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b: Bootstrap starting.
I20260806 06:36:11.973106 8596 tablet_bootstrap.cc:492] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18: Bootstrap starting.
I20260806 06:36:11.973148 8600 tablet_bootstrap.cc:492] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202: No bootstrap required, opened a new log
I20260806 06:36:11.973189 8600 ts_tablet_manager.cc:1397] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:11.973306 8600 raft_consensus.cc:348] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:11.973412 8600 raft_consensus.cc:374] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:11.973435 8600 raft_consensus.cc:729] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e615fbf098747529b61f02005270202, State: Initialized, Role: FOLLOWER
I20260806 06:36:11.973523 8596 tablet_bootstrap.cc:654] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:11.973500 8600 consensus_queue.cc:260] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:11.973610 8606 tablet_bootstrap.cc:654] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:11.973668 8600 ts_tablet_manager.cc:1428] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:11.974169 8596 tablet_bootstrap.cc:492] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18: No bootstrap required, opened a new log
I20260806 06:36:11.974227 8596 ts_tablet_manager.cc:1397] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:11.974376 8596 raft_consensus.cc:348] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:11.974455 8596 raft_consensus.cc:374] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:11.974478 8596 raft_consensus.cc:729] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270b539b1e414d63952194c491296c18, State: Initialized, Role: FOLLOWER
I20260806 06:36:11.974514 8606 tablet_bootstrap.cc:492] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b: No bootstrap required, opened a new log
I20260806 06:36:11.974527 8596 consensus_queue.cc:260] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:11.974572 8606 ts_tablet_manager.cc:1397] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:36:11.974745 8596 ts_tablet_manager.cc:1428] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:11.974753 8606 raft_consensus.cc:348] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:11.974833 8606 raft_consensus.cc:374] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:11.974871 8606 raft_consensus.cc:729] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 523ec1cf6a9e482a8fa0874e25825d9b, State: Initialized, Role: FOLLOWER
I20260806 06:36:11.974941 8606 consensus_queue.cc:260] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:11.975015 8612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.973997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.131:50937)
0806 06:36:11.974909 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.974955 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":886,"thread_start_us":87,"threads_started":1}
I20260806 06:36:11.975049 8462 heartbeater.cc:503] Master 127.29.142.189:41567 was elected leader, sending a full tablet report...
I20260806 06:36:11.975131 8606 ts_tablet_manager.cc:1428] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:11.975453 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.974859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.130:45269)
0806 06:36:11.975381 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.975420 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":86,"threads_started":1}
I20260806 06:36:11.975497 8612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.975217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.131:39131)
0806 06:36:11.975433 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.975463 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":82,"threads_started":1}
I20260806 06:36:11.975942 8265 heartbeater.cc:503] Master 127.29.142.189:41567 was elected leader, sending a full tablet report...
I20260806 06:36:11.975996 8068 heartbeater.cc:503] Master 127.29.142.189:41567 was elected leader, sending a full tablet report...
I20260806 06:36:11.976106 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.976012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.130:49169)
0806 06:36:11.976053 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.976077 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:36:11.976560 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.975341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.129:58527)
0806 06:36:11.976492 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.976527 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1115,"thread_start_us":83,"threads_started":1}
I20260806 06:36:11.976878 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:11.976779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.129:33475)
0806 06:36:11.976826 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:11.976849 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:36:12.028745 8613 raft_consensus.cc:482] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:36:12.028874 8613 raft_consensus.cc:504] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:12.029250 8613 leader_election.cc:288] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e615fbf098747529b61f02005270202 (127.29.142.131:34625), 523ec1cf6a9e482a8fa0874e25825d9b (127.29.142.129:41849)
I20260806 06:36:12.033352 8593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.029563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:46739 (local address 127.29.142.131:34625)
0806 06:36:12.029615 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:12.029617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:12.029891 (+ 274us) server_negotiation.cc:407] Connection header received
0806 06:36:12.029944 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:12.029945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:12.029953 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:12.030026 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:12.030167 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:12.030891 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:12.032404 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:12.033001 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:12.033065 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:12.033075 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:12.033233 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 06:36:12.033276 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:36:12.033352 8592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.029626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:47055 (local address 127.29.142.129:41849)
0806 06:36:12.029681 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:12.029684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:12.029694 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:36:12.029729 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:12.029730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:12.029744 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:12.029823 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:12.030014 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:12.030822 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:12.032404 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:12.032961 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:12.033047 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:12.033060 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:12.033234 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 06:36:12.033275 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:36:12.033401 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.029482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:41849 (local address 127.29.142.130:47055)
0806 06:36:12.029540 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:12.029554 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:12.029622 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:12.029843 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:12.029845 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:12.029849 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:12.029930 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:12.029933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:12.030896 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:12.030896 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:12.032302 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:12.032306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:12.033060 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:12.033061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:12.033134 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:12.033139 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:36:12.033202 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:36:12.033260 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":37}
I20260806 06:36:12.033401 8619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.029488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:34625 (local address 127.29.142.130:46739)
0806 06:36:12.029812 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:12.029831 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:12.029883 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:12.030038 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:12.030039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:12.030040 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:12.030091 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:12.030092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:12.030961 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:12.030962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:12.032309 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:12.032311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:12.033080 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:12.033081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:12.033142 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:12.033145 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:36:12.033202 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:36:12.033260 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":178,"thread_start_us":153,"threads_started":1}
I20260806 06:36:12.033715 8003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d2b8bbec434b8e81f64d40037b38d6" candidate_uuid: "270b539b1e414d63952194c491296c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" is_pre_election: true
I20260806 06:36:12.033825 8003 raft_consensus.cc:2454] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 270b539b1e414d63952194c491296c18 in term 0.
I20260806 06:36:12.034026 8389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d2b8bbec434b8e81f64d40037b38d6" candidate_uuid: "270b539b1e414d63952194c491296c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e615fbf098747529b61f02005270202" is_pre_election: true
I20260806 06:36:12.034085 8102 leader_election.cc:302] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 270b539b1e414d63952194c491296c18, 523ec1cf6a9e482a8fa0874e25825d9b; no voters:
I20260806 06:36:12.034162 8389 raft_consensus.cc:2454] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 270b539b1e414d63952194c491296c18 in term 0.
I20260806 06:36:12.034241 8613 raft_consensus.cc:2781] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:36:12.034289 8613 raft_consensus.cc:482] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:36:12.034319 8613 raft_consensus.cc:3037] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:12.035212 8613 raft_consensus.cc:504] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:12.035380 8613 leader_election.cc:288] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [CANDIDATE]: Term 1 election: Requested vote from peers 4e615fbf098747529b61f02005270202 (127.29.142.131:34625), 523ec1cf6a9e482a8fa0874e25825d9b (127.29.142.129:41849)
I20260806 06:36:12.035537 8389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d2b8bbec434b8e81f64d40037b38d6" candidate_uuid: "270b539b1e414d63952194c491296c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e615fbf098747529b61f02005270202"
I20260806 06:36:12.035578 8003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d2b8bbec434b8e81f64d40037b38d6" candidate_uuid: "270b539b1e414d63952194c491296c18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "523ec1cf6a9e482a8fa0874e25825d9b"
I20260806 06:36:12.035635 8389 raft_consensus.cc:3037] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:12.035647 8003 raft_consensus.cc:3037] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:12.036486 8389 raft_consensus.cc:2454] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 270b539b1e414d63952194c491296c18 in term 1.
I20260806 06:36:12.036486 8003 raft_consensus.cc:2454] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 270b539b1e414d63952194c491296c18 in term 1.
I20260806 06:36:12.036689 8102 leader_election.cc:302] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 270b539b1e414d63952194c491296c18, 523ec1cf6a9e482a8fa0874e25825d9b; no voters:
I20260806 06:36:12.036801 8613 raft_consensus.cc:2781] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:12.037019 8613 raft_consensus.cc:686] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 1 LEADER]: Becoming Leader. State: Replica: 270b539b1e414d63952194c491296c18, State: Running, Role: LEADER
I20260806 06:36:12.037111 8613 consensus_queue.cc:237] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } } peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } }
I20260806 06:36:12.037406 8619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.037270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.130:41159)
0806 06:36:12.037337 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:12.037374 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:36:12.037650 8463 tablet.cc:2369] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:36:12.037978 7556 catalog_manager.cc:5495] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 270b539b1e414d63952194c491296c18 (127.29.142.130). New cstate: current_term: 1 leader_uuid: "270b539b1e414d63952194c491296c18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "270b539b1e414d63952194c491296c18" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 } health_report { overall_health: UNKNOWN } } }
I20260806 06:36:12.063098 8591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.060557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58112 (local address 127.29.142.130:39021)
0806 06:36:12.060632 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:12.060634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:12.060647 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:36:12.060699 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:12.060700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:12.060727 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:12.060816 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:36:12.060961 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:12.061473 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:12.062244 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:12.062447 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:12.062524 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:12.062723 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:36:12.062836 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:36:12.062917 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:12.062974 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 06:36:12.062998 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:36:12.064600 8003 raft_consensus.cc:1261] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b [term 1 FOLLOWER]: Refusing update from remote peer 270b539b1e414d63952194c491296c18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:12.064687 8389 raft_consensus.cc:1261] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202 [term 1 FOLLOWER]: Refusing update from remote peer 270b539b1e414d63952194c491296c18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:12.064973 8620 consensus_queue.cc:1035] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "523ec1cf6a9e482a8fa0874e25825d9b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:12.065152 8613 consensus_queue.cc:1035] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e615fbf098747529b61f02005270202" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:12.066023 8617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.065864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.131:50045)
0806 06:36:12.065928 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:12.065985 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:36:12.067122 8625 mvcc.cc:205] Tried to move back new op lower bound from 7315448512773373952 to 7315448512664854528. Current Snapshot: MvccSnapshot[applied={T|T < 7315448512773373952}]
I20260806 06:36:12.067358 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.066785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.129:40615)
0806 06:36:12.067276 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:12.067317 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469}
I20260806 06:36:12.067397 8619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.067283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.130:54665)
0806 06:36:12.067338 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:12.067366 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:36:12.068200 8617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.067959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.131:50321)
0806 06:36:12.068136 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:12.068169 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260806 06:36:12.068506 8618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.068411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.129:52499)
0806 06:36:12.068450 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:12.068476 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:36:12.068786 8627 mvcc.cc:205] Tried to move back new op lower bound from 7315448512773373952 to 7315448512664854528. Current Snapshot: MvccSnapshot[applied={T|T < 7315448512773373952}]
I20260806 06:36:12.070725 8619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.070551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39715 (local address 127.29.142.130:49669)
0806 06:36:12.070641 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:12.070692 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:39715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 06:36:12.070820 7556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:12.070923 7556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:12.072489 7556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7619e678ceb947c49d8eafffa4b13272: Sending DeleteTablet for 3 replicas of tablet 66d2b8bbec434b8e81f64d40037b38d6
I20260806 06:36:12.072870 8160 tablet_service.cc:1514] Processing DeleteTablet for tablet 66d2b8bbec434b8e81f64d40037b38d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:12 UTC) from {username='oryx', principal='oryx/127.29.142.189@KRBTEST.COM'} at 127.0.0.1:58110
I20260806 06:36:12.072872 7975 tablet_service.cc:1514] Processing DeleteTablet for tablet 66d2b8bbec434b8e81f64d40037b38d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:12 UTC) from {username='oryx', principal='oryx/127.29.142.189@KRBTEST.COM'} at 127.0.0.1:40594
I20260806 06:36:12.073210 8638 tablet_replica.cc:330] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18: stopping tablet replica
I20260806 06:36:12.073246 8358 tablet_service.cc:1514] Processing DeleteTablet for tablet 66d2b8bbec434b8e81f64d40037b38d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:12 UTC) from {username='oryx', principal='oryx/127.29.142.189@KRBTEST.COM'} at 127.0.0.1:52812
I20260806 06:36:12.073283 8638 raft_consensus.cc:2229] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:36:12.073392 8638 raft_consensus.cc:2258] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:12.073524 8640 tablet_replica.cc:330] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b: stopping tablet replica
I20260806 06:36:12.073589 8640 raft_consensus.cc:2229] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:12.073640 8640 raft_consensus.cc:2258] T 66d2b8bbec434b8e81f64d40037b38d6 P 523ec1cf6a9e482a8fa0874e25825d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:12.073738 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 7754
I20260806 06:36:12.073792 8638 ts_tablet_manager.cc:1905] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:12.076642 8638 ts_tablet_manager.cc:1918] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:12.076706 8638 log.cc:1196] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/66d2b8bbec434b8e81f64d40037b38d6
I20260806 06:36:12.076952 8638 ts_tablet_manager.cc:1939] T 66d2b8bbec434b8e81f64d40037b38d6 P 270b539b1e414d63952194c491296c18: Deleting consensus metadata
I20260806 06:36:12.077292 7528 catalog_manager.cc:4841] TS 270b539b1e414d63952194c491296c18 (127.29.142.130:39021): tablet 66d2b8bbec434b8e81f64d40037b38d6 (table test-table [id=f203923a28264adda9acb2298e9f12e7]) successfully deleted
I20260806 06:36:12.073454 8641 tablet_replica.cc:330] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202: stopping tablet replica
I20260806 06:36:12.077487 8641 raft_consensus.cc:2229] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:12.077546 8641 raft_consensus.cc:2258] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:12.078318 8641 ts_tablet_manager.cc:1905] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:12.080102 8641 ts_tablet_manager.cc:1918] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:12.080168 8641 log.cc:1196] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/66d2b8bbec434b8e81f64d40037b38d6
I20260806 06:36:12.080391 8641 ts_tablet_manager.cc:1939] T 66d2b8bbec434b8e81f64d40037b38d6 P 4e615fbf098747529b61f02005270202: Deleting consensus metadata
I20260806 06:36:12.090168 7526 catalog_manager.cc:4841] TS 4e615fbf098747529b61f02005270202 (127.29.142.131:34625): tablet 66d2b8bbec434b8e81f64d40037b38d6 (table test-table [id=f203923a28264adda9acb2298e9f12e7]) successfully deleted
I20260806 06:36:12.097074 8565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:12.096838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:41849 (local address 127.0.0.1:40604)
0806 06:36:12.096930 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:12.096992 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:41849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 06:36:12.097141 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 8076
W20260806 06:36:12.103834 7527 catalog_manager.cc:4584] TS 523ec1cf6a9e482a8fa0874e25825d9b (127.29.142.129:41849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 66d2b8bbec434b8e81f64d40037b38d6: Network error: Client connection negotiation failed: client connection to 127.29.142.129:41849: connect: Connection refused (error 111)
I20260806 06:36:12.110931 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 8272
I20260806 06:36:12.126878 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 7356
I20260806 06:36:12.137794 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 7626
2026-08-06T06:36:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4948 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:36:12 dist-test-slave-3kk6 krb5kdc[8645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:36:12 dist-test-slave-3kk6 krb5kdc[8645](info): set up 2 sockets
Aug 06 06:36:12 dist-test-slave-3kk6 krb5kdc[8645](info): commencing operation
krb5kdc: starting...
W20260806 06:36:13.362991 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:36:13 dist-test-slave-3kk6 krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:36:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.190.keytab.
Entry for principal oryx/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.190.keytab.
I20260806 06:36:13.493539 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:37211
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.190.keytab
--principal=oryx/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:37211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:13.581131 8663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:13.581427 8663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:13.581475 8663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:13.582839 8663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:13.582917 8663 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:36:13.582939 8663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:13.582962 8663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:13.582993 8663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:13.583024 8663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:13.584746 8663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:37211
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.190.keytab
--principal=oryx/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:37211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:13.585080 8663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:13.585342 8663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:13.590487 8670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:13.590536 8669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:13.590566 8673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:13.590883 8663 server_base.cc:1034] running on GCE node
I20260806 06:36:13.591127 8663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:13.591403 8663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:13.592573 8663 hybrid_clock.cc:648] HybridClock initialized: now 1785998173592570 us; error 44 us; skew 500 ppm
Aug 06 06:36:13 dist-test-slave-3kk6 krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998173, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:13.595306 8663 init.cc:377] Logged in from keytab as oryx/127.29.142.190@KRBTEST.COM (short username oryx)
I20260806 06:36:13.595793 8663 webserver.cc:460] Webserver started at http://127.29.142.190:44233/ using document root <none> and password file <none>
I20260806 06:36:13.596060 8663 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:13.596131 8663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:13.596295 8663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:13.597461 8663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "8f3935f5d44f4420b8b7e6ff954a82e5"
format_stamp: "Formatted at 2026-08-06 06:36:13 on dist-test-slave-3kk6"
server_key: "e4ddcfc7d6306ceee5a875837c510311"
server_key_iv: "72b05af3bbfe0abc1c12a879d7715b72"
server_key_version: "encryptionkey@0"
I20260806 06:36:13.597872 8663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "8f3935f5d44f4420b8b7e6ff954a82e5"
format_stamp: "Formatted at 2026-08-06 06:36:13 on dist-test-slave-3kk6"
server_key: "e4ddcfc7d6306ceee5a875837c510311"
server_key_iv: "72b05af3bbfe0abc1c12a879d7715b72"
server_key_version: "encryptionkey@0"
I20260806 06:36:13.599498 8663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:13.600343 8684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:13.600525 8663 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:13.600636 8663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "8f3935f5d44f4420b8b7e6ff954a82e5"
format_stamp: "Formatted at 2026-08-06 06:36:13 on dist-test-slave-3kk6"
server_key: "e4ddcfc7d6306ceee5a875837c510311"
server_key_iv: "72b05af3bbfe0abc1c12a879d7715b72"
server_key_version: "encryptionkey@0"
I20260806 06:36:13.600720 8663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:13.628321 8663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:13.630620 8663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:13.630864 8663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:13.635805 8663 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:37211
I20260806 06:36:13.635864 8781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:37211 every 8 connection(s)
I20260806 06:36:13.636250 8663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:13.637075 8782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:13.639432 8782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5: Bootstrap starting.
I20260806 06:36:13.640061 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 8663
I20260806 06:36:13.640198 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:13.640198 8782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:13.640434 30266 external_mini_cluster.cc:1428] Setting key cef7e5edfc1a46c4cf825fa9567b293b
I20260806 06:36:13.640547 8782 log.cc:824] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:13.641367 8782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5: No bootstrap required, opened a new log
I20260806 06:36:13.643493 8782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3935f5d44f4420b8b7e6ff954a82e5" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37211 } }
I20260806 06:36:13.643656 8782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:13.643704 8782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3935f5d44f4420b8b7e6ff954a82e5, State: Initialized, Role: FOLLOWER
I20260806 06:36:13.643880 8782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3935f5d44f4420b8b7e6ff954a82e5" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37211 } }
I20260806 06:36:13.643975 8782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:13.644023 8782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:13.644078 8782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:13.644658 8782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3935f5d44f4420b8b7e6ff954a82e5" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37211 } }
I20260806 06:36:13.644791 8782 leader_election.cc:302] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f3935f5d44f4420b8b7e6ff954a82e5; no voters:
I20260806 06:36:13.644979 8782 leader_election.cc:288] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:13.645031 8788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:13.645187 8788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [term 1 LEADER]: Becoming Leader. State: Replica: 8f3935f5d44f4420b8b7e6ff954a82e5, State: Running, Role: LEADER
I20260806 06:36:13.645254 8782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:13.645287 8788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3935f5d44f4420b8b7e6ff954a82e5" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37211 } }
Aug 06 06:36:13 dist-test-slave-3kk6 krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
I20260806 06:36:13.646961 8791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f3935f5d44f4420b8b7e6ff954a82e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3935f5d44f4420b8b7e6ff954a82e5" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37211 } } }
I20260806 06:36:13.647101 8789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f3935f5d44f4420b8b7e6ff954a82e5. Latest consensus state: current_term: 1 leader_uuid: "8f3935f5d44f4420b8b7e6ff954a82e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3935f5d44f4420b8b7e6ff954a82e5" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37211 } } }
W20260806 06:36:13.647289 8807 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:36:13.647809 8807 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:36:13.647783 8791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:13.647797 8789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:13.648036 8810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:13.648231 8810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:13.650138 8810 catalog_manager.cc:1347] Generated new cluster ID: 4b8eb097110b4501bcf28f2015f3ad4d
I20260806 06:36:13.650218 8810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:36:13.652078 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:13.641030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45878 (local address 127.29.142.190:37211)
0806 06:36:13.641215 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:13.641226 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:13.641262 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:36:13.641732 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:13.641740 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:13.641790 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:13.641940 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:13.642597 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.643535 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.644478 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.645003 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.645524 (+ 521us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:13.648217 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:13.648222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:13.648222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:13.648237 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:13.650397 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:13.650854 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:13.650857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:13.650858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:13.650955 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:13.651171 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:13.651171 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:13.651172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:13.651421 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:13.651580 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:13.651706 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 06:36:13.651859 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260806 06:36:13.661586 8810 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:13.662367 8810 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:13.675529 8810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5: Generated new TSK 0
I20260806 06:36:13.675786 8810 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab.
Entry for principal oryx/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab.
I20260806 06:36:13.705314 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37211
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab
--principal=oryx/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:13.790855 8815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:13.791096 8815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:13.791141 8815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:13.792316 8815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:13.792382 8815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:13.792418 8815 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:36:13.792469 8815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:36:13.794016 8815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab
--principal=oryx/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.142.190:37211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:13.794296 8815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:13.794528 8815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:13.799695 8824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:13.799697 8827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:13.799724 8823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:13.799963 8815 server_base.cc:1034] running on GCE node
I20260806 06:36:13.800252 8815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:13.800529 8815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:13.801712 8815 hybrid_clock.cc:648] HybridClock initialized: now 1785998173801699 us; error 38 us; skew 500 ppm
Aug 06 06:36:13 dist-test-slave-3kk6 krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998173, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:13.804572 8815 init.cc:377] Logged in from keytab as oryx/127.29.142.129@KRBTEST.COM (short username oryx)
I20260806 06:36:13.805114 8815 webserver.cc:460] Webserver started at http://127.29.142.129:46619/ using document root <none> and password file <none>
I20260806 06:36:13.805379 8815 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:13.805447 8815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:13.805604 8815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:13.806800 8815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "87816cb844af4ed5a225b36dfa101298"
format_stamp: "Formatted at 2026-08-06 06:36:13 on dist-test-slave-3kk6"
server_key: "e31d39ce9dab330837a89fe62bcfdc63"
server_key_iv: "da79ddb186286502809429d56d0e5260"
server_key_version: "encryptionkey@0"
I20260806 06:36:13.807178 8815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "87816cb844af4ed5a225b36dfa101298"
format_stamp: "Formatted at 2026-08-06 06:36:13 on dist-test-slave-3kk6"
server_key: "e31d39ce9dab330837a89fe62bcfdc63"
server_key_iv: "da79ddb186286502809429d56d0e5260"
server_key_version: "encryptionkey@0"
I20260806 06:36:13.808585 8815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:36:13.809406 8836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:13.809597 8815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:13.809701 8815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "87816cb844af4ed5a225b36dfa101298"
format_stamp: "Formatted at 2026-08-06 06:36:13 on dist-test-slave-3kk6"
server_key: "e31d39ce9dab330837a89fe62bcfdc63"
server_key_iv: "da79ddb186286502809429d56d0e5260"
server_key_version: "encryptionkey@0"
I20260806 06:36:13.809792 8815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:13.819367 8815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:13.821367 8815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:13.821524 8815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:13.822041 8815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:13.822103 8815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:13.822145 8815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:13.822172 8815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:13.829236 8815 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:46751
I20260806 06:36:13.829310 9000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:46751 every 8 connection(s)
Aug 06 06:36:13 dist-test-slave-3kk6 krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998173, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
I20260806 06:36:13.829694 8815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:36:13.831228 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 8815
I20260806 06:36:13.831328 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:36:13.831542 30266 external_mini_cluster.cc:1428] Setting key c93713e4b78119221d82b5cc01e5f649
I20260806 06:36:13.833134 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:13.822542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:35455 (local address 127.29.142.190:37211)
0806 06:36:13.822616 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:13.822619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:13.822817 (+ 198us) server_negotiation.cc:407] Connection header received
0806 06:36:13.823352 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:13.823353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:13.823368 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:13.823445 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:13.824419 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.825296 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.826877 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.827218 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.827301 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:13.830148 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:13.830150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:13.830151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:13.830160 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:13.831555 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:13.832079 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:13.832081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:13.832082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:13.832153 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:13.832485 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:13.832486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:13.832486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:13.832599 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:13.832656 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:13.832995 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 06:36:13.833070 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:36:13.833457 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:13.822488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37211 (local address 127.29.142.129:35455)
0806 06:36:13.822673 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:13.822715 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:13.823245 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:13.823565 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:13.823574 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:13.823937 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:13.824258 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:13.824264 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.825399 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.825400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:13.826780 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:13.826784 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.827310 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.827311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:13.827455 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:13.828067 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:13.828080 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:13.830002 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:13.831679 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:13.831685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:13.831687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:13.831999 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:13.832230 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:13.832231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:13.832231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:13.832405 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:13.832668 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:13.832675 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:13.832846 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 06:36:13.832974 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 06:36:13.833167 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260806 06:36:13.834036 9001 heartbeater.cc:347] Connected to a master server at 127.29.142.190:37211
I20260806 06:36:13.834139 9001 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:13.834321 9001 heartbeater.cc:511] Master 127.29.142.190:37211 requested a full tablet report, sending...
I20260806 06:36:13.834913 8709 ts_manager.cc:194] Registered new tserver with Master: 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751)
I20260806 06:36:13.835852 8709 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:35455
WARNING: no policy specified for oryx/127.29.142.130@KRBTEST.COM; defaulting to no policy
I20260806 06:36:13.841711 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:13.835202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37211 (local address 127.29.142.129:42181)
0806 06:36:13.835425 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:13.835437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:13.835498 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:13.835631 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:13.835632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:13.835836 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:13.835910 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:13.835912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.836851 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.836852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:13.837344 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:13.837346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.837885 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.837886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:13.837937 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:13.838446 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:13.838451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:13.838863 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:13.840662 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:13.840663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:13.840663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:13.840846 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:13.841069 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:13.841070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:13.841070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:13.841122 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:13.841393 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:13.841393 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:13.841429 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 06:36:13.841489 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:36:13.841572 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":156,"threads_started":1}
I20260806 06:36:13.841753 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:13.835259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:42181 (local address 127.29.142.190:37211)
0806 06:36:13.835321 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:13.835323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:13.835489 (+ 166us) server_negotiation.cc:407] Connection header received
0806 06:36:13.835548 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:13.835549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:13.835560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:13.835625 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:13.836016 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.836713 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.837424 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.837780 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.837869 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:13.838947 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:13.838949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:13.838949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:13.838957 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:13.840575 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:13.840925 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:13.840926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:13.840927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:13.840989 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:13.841181 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:13.841182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:13.841182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:13.841328 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:13.841405 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:13.841538 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 06:36:13.841630 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab.
Entry for principal oryx/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab.
I20260806 06:36:13.861366 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37211
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab
--principal=oryx/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:13.950762 9016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:13.951025 9016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:13.951052 9016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:13.952299 9016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:13.952368 9016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:13.952390 9016 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:36:13.952447 9016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:36:13.954097 9016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab
--principal=oryx/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.142.190:37211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:13.954378 9016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:13.954640 9016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:13.960220 9025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:13.960264 9016 server_base.cc:1034] running on GCE node
W20260806 06:36:13.960268 9028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:13.960225 9024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:13.960695 9016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:13.960935 9016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:13.962098 9016 hybrid_clock.cc:648] HybridClock initialized: now 1785998173962078 us; error 42 us; skew 500 ppm
Aug 06 06:36:13 dist-test-slave-3kk6 krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998173, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:13.964913 9016 init.cc:377] Logged in from keytab as oryx/127.29.142.130@KRBTEST.COM (short username oryx)
I20260806 06:36:13.965452 9016 webserver.cc:460] Webserver started at http://127.29.142.130:42629/ using document root <none> and password file <none>
I20260806 06:36:13.965724 9016 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:13.965794 9016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:13.965946 9016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:13.967149 9016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "98116dbe143a48979daa6ce5b8499588"
format_stamp: "Formatted at 2026-08-06 06:36:13 on dist-test-slave-3kk6"
server_key: "06ce4e678e9f7badc3ece1985545e7c2"
server_key_iv: "644891ae99d17f135f542eede07d6eb2"
server_key_version: "encryptionkey@0"
I20260806 06:36:13.967532 9016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "98116dbe143a48979daa6ce5b8499588"
format_stamp: "Formatted at 2026-08-06 06:36:13 on dist-test-slave-3kk6"
server_key: "06ce4e678e9f7badc3ece1985545e7c2"
server_key_iv: "644891ae99d17f135f542eede07d6eb2"
server_key_version: "encryptionkey@0"
I20260806 06:36:13.968935 9016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:36:13.969674 9039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:13.969846 9016 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:13.969941 9016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "98116dbe143a48979daa6ce5b8499588"
format_stamp: "Formatted at 2026-08-06 06:36:13 on dist-test-slave-3kk6"
server_key: "06ce4e678e9f7badc3ece1985545e7c2"
server_key_iv: "644891ae99d17f135f542eede07d6eb2"
server_key_version: "encryptionkey@0"
I20260806 06:36:13.970021 9016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:13.978542 9016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:13.980572 9016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:13.980767 9016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:13.981389 9016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:13.981447 9016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:13.981483 9016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:13.981513 9016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:36:13 dist-test-slave-3kk6 krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998173, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
I20260806 06:36:13.989395 9016 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:42911
I20260806 06:36:13.989488 9220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:42911 every 8 connection(s)
I20260806 06:36:13.989872 9016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:36:13.991827 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:13.981849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:50263 (local address 127.29.142.190:37211)
0806 06:36:13.981930 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:13.981933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:13.982145 (+ 212us) server_negotiation.cc:407] Connection header received
0806 06:36:13.982675 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:13.982676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:13.982693 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:13.982788 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:13.983459 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.983976 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.985017 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.985225 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.985312 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:13.988709 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:13.988710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:13.988711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:13.988720 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:13.990523 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:13.990990 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:13.990991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:13.990992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:13.991051 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:13.991271 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:13.991272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:13.991272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:13.991385 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:13.991438 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:13.991692 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 06:36:13.991760 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:36:13.992098 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:13.981815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37211 (local address 127.29.142.130:50263)
0806 06:36:13.982005 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:13.982049 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:13.982549 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:13.982875 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:13.982880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:13.983150 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:13.983351 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:13.983356 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.984074 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.984075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:13.984923 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:13.984925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.985332 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.985333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:13.985495 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:13.986110 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:13.986123 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:13.988594 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:13.990615 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:13.990620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:13.990621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:13.990911 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:13.991114 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:13.991115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:13.991115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:13.991217 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:13.991438 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:13.991443 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:13.991564 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 06:36:13.991658 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 06:36:13.991799 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260806 06:36:13.992705 9221 heartbeater.cc:347] Connected to a master server at 127.29.142.190:37211
I20260806 06:36:13.992796 9221 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:13.993006 9221 heartbeater.cc:511] Master 127.29.142.190:37211 requested a full tablet report, sending...
I20260806 06:36:13.993479 8708 ts_manager.cc:194] Registered new tserver with Master: 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911)
I20260806 06:36:13.994004 8708 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:50263
I20260806 06:36:13.997545 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 9016
I20260806 06:36:13.997710 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:36:13.997973 30266 external_mini_cluster.cc:1428] Setting key 2ce4644da4b55187e9c6cbb27f6fcde8
I20260806 06:36:14.000073 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:13.993840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:56803 (local address 127.29.142.190:37211)
0806 06:36:13.993911 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:13.993913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:13.994040 (+ 127us) server_negotiation.cc:407] Connection header received
0806 06:36:13.994068 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:13.994069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:13.994081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:13.994147 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:13.994581 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.995128 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.995815 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.995991 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.996043 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:13.997098 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:13.997099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:13.997100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:13.997106 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:13.998843 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:13.999196 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:13.999198 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:13.999198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:13.999264 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:13.999479 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:13.999480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:13.999480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:13.999636 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:13.999716 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:13.999884 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 06:36:13.999966 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:36:14.000084 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:13.993773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37211 (local address 127.29.142.130:56803)
0806 06:36:13.993970 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:13.993980 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:13.994032 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:13.994196 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:13.994198 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:13.994414 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:13.994496 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:13.994499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.995230 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.995231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:13.995742 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:13.995743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:13.996068 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:13.996069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:13.996151 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:13.996694 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:13.996700 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:13.997014 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:13.998920 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:13.998921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:13.998922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:13.999118 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:13.999343 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:13.999344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:13.999344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:13.999408 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:13.999714 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:13.999715 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:13.999767 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 06:36:13.999845 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:36:13.999969 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for oryx/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab.
Entry for principal oryx/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab.
I20260806 06:36:14.026086 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37211
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab
--principal=oryx/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:14.122099 9234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:14.122352 9234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:14.122377 9234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:14.123692 9234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:14.123747 9234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:14.123768 9234 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:36:14.123786 9234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:36:14.125311 9234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab
--principal=oryx/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.142.190:37211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:14.125514 9234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:14.125713 9234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:14.131007 9244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:14.131021 9241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:14.131016 9242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:14.131052 9234 server_base.cc:1034] running on GCE node
I20260806 06:36:14.131448 9234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:14.131732 9234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:14.132893 9234 hybrid_clock.cc:648] HybridClock initialized: now 1785998174132877 us; error 41 us; skew 500 ppm
Aug 06 06:36:14 dist-test-slave-3kk6 krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998174, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:14.135864 9234 init.cc:377] Logged in from keytab as oryx/127.29.142.131@KRBTEST.COM (short username oryx)
I20260806 06:36:14.136350 9234 webserver.cc:460] Webserver started at http://127.29.142.131:46493/ using document root <none> and password file <none>
I20260806 06:36:14.136583 9234 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:14.136631 9234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:14.136806 9234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:14.137969 9234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "7c9e048616d347d7aafc8191fc2e9838"
format_stamp: "Formatted at 2026-08-06 06:36:14 on dist-test-slave-3kk6"
server_key: "58bf1d0b6763f13ca932e020d4f56ffe"
server_key_iv: "ba57707202961e8be5e247ae1bbe132b"
server_key_version: "encryptionkey@0"
I20260806 06:36:14.138358 9234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "7c9e048616d347d7aafc8191fc2e9838"
format_stamp: "Formatted at 2026-08-06 06:36:14 on dist-test-slave-3kk6"
server_key: "58bf1d0b6763f13ca932e020d4f56ffe"
server_key_iv: "ba57707202961e8be5e247ae1bbe132b"
server_key_version: "encryptionkey@0"
I20260806 06:36:14.139914 9234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:14.140697 9257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:14.140894 9234 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:14.140995 9234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "7c9e048616d347d7aafc8191fc2e9838"
format_stamp: "Formatted at 2026-08-06 06:36:14 on dist-test-slave-3kk6"
server_key: "58bf1d0b6763f13ca932e020d4f56ffe"
server_key_iv: "ba57707202961e8be5e247ae1bbe132b"
server_key_version: "encryptionkey@0"
I20260806 06:36:14.141076 9234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:14.158612 9234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:14.160670 9234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:14.160856 9234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:14.161459 9234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:14.161530 9234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:14.161579 9234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:14.161605 9234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:14.169457 9234 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:41265
I20260806 06:36:14.169546 9416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:41265 every 8 connection(s)
Aug 06 06:36:14 dist-test-slave-3kk6 krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998174, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
I20260806 06:36:14.169920 9234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:36:14.173153 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.161962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:44167 (local address 127.29.142.190:37211)
0806 06:36:14.162035 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.162038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.162315 (+ 277us) server_negotiation.cc:407] Connection header received
0806 06:36:14.163124 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.163126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.163136 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.163182 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:14.164138 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.164650 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.166386 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.166731 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.166827 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.170297 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:14.170299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:14.170299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:14.170307 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:14.171914 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:14.172300 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:14.172301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:14.172302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:14.172364 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:14.172572 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:14.172573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:14.172573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:14.172676 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:14.172727 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.173014 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.173088 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:36:14.173318 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.161939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37211 (local address 127.29.142.131:44167)
0806 06:36:14.162147 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.162201 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.163005 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.163279 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.163283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.163716 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:14.164033 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.164041 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.164757 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.164758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.166286 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.166290 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.166861 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.166863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.167036 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.168289 (+ 1253us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:14.168302 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:14.170148 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:14.171991 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:14.171996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:14.171998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:14.172230 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:14.172421 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:14.172422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:14.172422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:14.172520 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:14.172729 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:14.172733 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:14.172898 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.172986 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.173114 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":102,"threads_started":1}
I20260806 06:36:14.173894 9417 heartbeater.cc:347] Connected to a master server at 127.29.142.190:37211
I20260806 06:36:14.173933 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 9234
I20260806 06:36:14.173992 9417 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:14.174021 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:36:14.174202 9417 heartbeater.cc:511] Master 127.29.142.190:37211 requested a full tablet report, sending...
I20260806 06:36:14.174224 30266 external_mini_cluster.cc:1428] Setting key 729537214d49db168318ca0afedf45d4
I20260806 06:36:14.174640 8708 ts_manager.cc:194] Registered new tserver with Master: 7c9e048616d347d7aafc8191fc2e9838 (127.29.142.131:41265)
I20260806 06:36:14.175172 8708 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:44167
I20260806 06:36:14.175275 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:36:14.181330 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.175026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37211 (local address 127.29.142.131:48765)
0806 06:36:14.175159 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.175171 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.175250 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.175395 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.175396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.175662 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:14.175731 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.175734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.176665 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.176665 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.177181 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.177182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.177573 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.177574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.177641 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.178187 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:14.178193 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:14.178511 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:14.180268 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:14.180270 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:14.180270 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:14.180454 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:14.180659 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:14.180660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:14.180660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:14.180722 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:14.180984 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:14.180985 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:14.181029 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.181110 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.181234 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260806 06:36:14.181327 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.175074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:48765 (local address 127.29.142.190:37211)
0806 06:36:14.175117 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.175119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.175232 (+ 113us) server_negotiation.cc:407] Connection header received
0806 06:36:14.175313 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.175313 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.175332 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.175389 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:14.175812 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.176567 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.177281 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.177485 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.177571 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.178602 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:14.178603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:14.178604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:14.178612 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:14.180173 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:14.180527 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:14.180528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:14.180529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:14.180592 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:14.180792 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:14.180792 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:14.180793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:14.180906 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:14.180978 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.181148 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.181225 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 06:36:14 dist-test-slave-3kk6 krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998174, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:36:14 dist-test-slave-3kk6 krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:14 dist-test-slave-3kk6 krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:36:14.198890 8786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:14.193275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45882 (local address 127.29.142.190:37211)
0806 06:36:14.193325 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.193327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.193336 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:36:14.193363 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.193364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.193373 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.193425 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:14.194038 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.194539 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.195350 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.195545 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.195615 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.198787 (+ 3172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45882: BlockingRecv error: recv got EOF from 127.0.0.1:45882 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 06:36:14 dist-test-slave-3kk6 krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998174, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
I20260806 06:36:14.206875 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.199759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45890 (local address 127.29.142.190:37211)
0806 06:36:14.199824 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.199826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.199866 (+ 40us) server_negotiation.cc:407] Connection header received
0806 06:36:14.199926 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.199927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.199935 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.199990 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:14.200481 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.200967 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.201630 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.201840 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.201899 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.204471 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:14.204472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:14.204472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:14.204479 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:14.205813 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:14.206174 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:14.206175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:14.206175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:14.206229 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:14.206399 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:14.206400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:14.206400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:14.206505 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:14.206584 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.206738 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.206804 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:36:14.208550 8708 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:36:14.210191 8708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:36:14.216727 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.212935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:42911 (local address 127.0.0.1:45408)
0806 06:36:14.213290 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.213300 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.213344 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.213608 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.213609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.213611 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:14.213660 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.213662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.214625 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.214626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.215540 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.215544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.216409 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.216410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.216492 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.216508 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.216583 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.216638 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":83,"threads_started":1}
I20260806 06:36:14.216758 9455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.213067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45408 (local address 127.29.142.130:42911)
0806 06:36:14.213449 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.213453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.213468 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:36:14.213501 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.213502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.213514 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.213598 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:14.213757 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.214535 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.215654 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.216303 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.216432 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.216497 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.216610 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.216649 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":103,"threads_started":1}
I20260806 06:36:14.217656 9448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.212644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:46751 (local address 127.0.0.1:53658)
0806 06:36:14.213196 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.213215 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.213316 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.213542 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.213545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.213548 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:14.213622 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.213624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.215024 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.215025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.216262 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.216265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.217303 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.217304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.217390 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.217394 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.217480 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.217527 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":48,"thread_start_us":125,"threads_started":1}
I20260806 06:36:14.217651 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.212771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34386 (local address 127.29.142.131:41265)
0806 06:36:14.212952 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.212958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.213175 (+ 217us) server_negotiation.cc:407] Connection header received
0806 06:36:14.213283 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.213286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.213301 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.213397 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:14.213816 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.214926 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.216378 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.217113 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.217210 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.217259 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.217503 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.217547 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260806 06:36:14.217592 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.212640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:41265 (local address 127.0.0.1:34386)
0806 06:36:14.213052 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.213099 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.213192 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.213692 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.213693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.213695 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:14.213729 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.213730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.215021 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.215023 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.216263 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.216265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.217296 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.217297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.217385 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.217389 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.217450 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.217491 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":51,"thread_start_us":123,"threads_started":1}
I20260806 06:36:14.217801 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.212864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53658 (local address 127.29.142.129:46751)
0806 06:36:14.213136 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.213141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.213283 (+ 142us) server_negotiation.cc:407] Connection header received
0806 06:36:14.213431 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.213434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.213451 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.213526 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:14.213719 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.214916 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.216359 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.217094 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.217205 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.217262 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.217633 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.217684 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":80,"threads_started":1}
I20260806 06:36:14.217819 9120 tablet_service.cc:1467] Processing CreateTablet for tablet 4d19d529bd2b48b5b2119a5d6fafeb71 (DEFAULT_TABLE table=test-table [id=b2ad39c8541e4edaaccf6876cf4070dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:14.218283 9120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d19d529bd2b48b5b2119a5d6fafeb71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:14.218452 9312 tablet_service.cc:1467] Processing CreateTablet for tablet 4d19d529bd2b48b5b2119a5d6fafeb71 (DEFAULT_TABLE table=test-table [id=b2ad39c8541e4edaaccf6876cf4070dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:14.218524 8904 tablet_service.cc:1467] Processing CreateTablet for tablet 4d19d529bd2b48b5b2119a5d6fafeb71 (DEFAULT_TABLE table=test-table [id=b2ad39c8541e4edaaccf6876cf4070dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:14.218813 9312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d19d529bd2b48b5b2119a5d6fafeb71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:14.218916 8904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d19d529bd2b48b5b2119a5d6fafeb71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:14.222050 9456 tablet_bootstrap.cc:492] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838: Bootstrap starting.
I20260806 06:36:14.222834 9457 tablet_bootstrap.cc:492] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588: Bootstrap starting.
I20260806 06:36:14.223039 9459 tablet_bootstrap.cc:492] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298: Bootstrap starting.
I20260806 06:36:14.223067 9456 tablet_bootstrap.cc:654] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:14.223526 9456 log.cc:824] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:14.223575 9457 tablet_bootstrap.cc:654] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:14.223601 9459 tablet_bootstrap.cc:654] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:14.223933 9457 log.cc:824] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:14.223970 9459 log.cc:824] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:14.224491 9456 tablet_bootstrap.cc:492] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838: No bootstrap required, opened a new log
I20260806 06:36:14.224573 9456 ts_tablet_manager.cc:1397] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:36:14.224713 9459 tablet_bootstrap.cc:492] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298: No bootstrap required, opened a new log
I20260806 06:36:14.224745 9457 tablet_bootstrap.cc:492] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588: No bootstrap required, opened a new log
I20260806 06:36:14.224794 9457 ts_tablet_manager.cc:1397] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:36:14.224794 9459 ts_tablet_manager.cc:1397] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:36:14.226454 9456 raft_consensus.cc:348] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } }
I20260806 06:36:14.226634 9456 raft_consensus.cc:374] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:14.226737 9456 raft_consensus.cc:729] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9e048616d347d7aafc8191fc2e9838, State: Initialized, Role: FOLLOWER
I20260806 06:36:14.226886 9456 consensus_queue.cc:260] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } }
I20260806 06:36:14.226895 9457 raft_consensus.cc:348] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } }
I20260806 06:36:14.226895 9459 raft_consensus.cc:348] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } }
I20260806 06:36:14.227058 9459 raft_consensus.cc:374] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:14.227058 9457 raft_consensus.cc:374] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:14.227094 9459 raft_consensus.cc:729] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87816cb844af4ed5a225b36dfa101298, State: Initialized, Role: FOLLOWER
I20260806 06:36:14.227094 9457 raft_consensus.cc:729] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98116dbe143a48979daa6ce5b8499588, State: Initialized, Role: FOLLOWER
I20260806 06:36:14.227211 9457 consensus_queue.cc:260] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } }
I20260806 06:36:14.227241 9456 ts_tablet_manager.cc:1428] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:36:14.227226 9459 consensus_queue.cc:260] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } }
I20260806 06:36:14.227380 9417 heartbeater.cc:503] Master 127.29.142.190:37211 was elected leader, sending a full tablet report...
I20260806 06:36:14.227478 9459 ts_tablet_manager.cc:1428] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:36:14.227478 9457 ts_tablet_manager.cc:1428] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:36:14.227602 9001 heartbeater.cc:503] Master 127.29.142.190:37211 was elected leader, sending a full tablet report...
I20260806 06:36:14.227655 9221 heartbeater.cc:503] Master 127.29.142.190:37211 was elected leader, sending a full tablet report...
W20260806 06:36:14.240505 9223 tablet.cc:2369] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:36:14.330698 9002 tablet.cc:2369] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:36:14.361644 9469 raft_consensus.cc:482] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:36:14.361851 9469 raft_consensus.cc:504] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } }
I20260806 06:36:14.362253 9469 leader_election.cc:288] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c9e048616d347d7aafc8191fc2e9838 (127.29.142.131:41265), 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911)
I20260806 06:36:14.365651 9455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.362602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:52963 (local address 127.29.142.130:42911)
0806 06:36:14.362773 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.362775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.362786 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:14.362876 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.362877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.362886 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.362950 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:14.363127 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.363909 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.364912 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.365282 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.365368 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.365378 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.365522 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.365561 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260806 06:36:14.365656 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.362619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:48225 (local address 127.29.142.131:41265)
0806 06:36:14.362717 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.362719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.362732 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:36:14.362853 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.362854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.362866 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.362901 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:14.363173 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.363916 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.364923 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.365288 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.365385 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.365394 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.365546 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.365585 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 06:36:14.365733 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.362467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:41265 (local address 127.29.142.129:48225)
0806 06:36:14.362629 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.362642 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.362730 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.362986 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.362987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.362990 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:14.363068 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.363070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.364007 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.364008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.364803 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.364805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.365384 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.365385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.365458 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.365462 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.365537 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.365580 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":1,"thread_start_us":97,"threads_started":1}
I20260806 06:36:14.365756 9472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.362476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:42911 (local address 127.29.142.129:52963)
0806 06:36:14.362693 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.362737 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.362801 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.362970 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.362970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.362973 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:14.363044 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.363046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.364053 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.364054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.364804 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.364806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.365368 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.365369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.365446 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.365450 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.365516 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.365561 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":3,"thread_start_us":67,"threads_started":1}
I20260806 06:36:14.366101 9352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d19d529bd2b48b5b2119a5d6fafeb71" candidate_uuid: "87816cb844af4ed5a225b36dfa101298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9e048616d347d7aafc8191fc2e9838" is_pre_election: true
I20260806 06:36:14.366230 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d19d529bd2b48b5b2119a5d6fafeb71" candidate_uuid: "87816cb844af4ed5a225b36dfa101298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98116dbe143a48979daa6ce5b8499588" is_pre_election: true
I20260806 06:36:14.366299 9352 raft_consensus.cc:2454] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87816cb844af4ed5a225b36dfa101298 in term 0.
I20260806 06:36:14.366374 9153 raft_consensus.cc:2454] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87816cb844af4ed5a225b36dfa101298 in term 0.
I20260806 06:36:14.366559 8841 leader_election.cc:302] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c9e048616d347d7aafc8191fc2e9838, 87816cb844af4ed5a225b36dfa101298; no voters:
I20260806 06:36:14.366732 9469 raft_consensus.cc:2781] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:36:14.366817 9469 raft_consensus.cc:482] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:36:14.366855 9469 raft_consensus.cc:3037] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:14.367641 9469 raft_consensus.cc:504] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } }
I20260806 06:36:14.367803 9469 leader_election.cc:288] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [CANDIDATE]: Term 1 election: Requested vote from peers 7c9e048616d347d7aafc8191fc2e9838 (127.29.142.131:41265), 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911)
I20260806 06:36:14.367965 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d19d529bd2b48b5b2119a5d6fafeb71" candidate_uuid: "87816cb844af4ed5a225b36dfa101298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98116dbe143a48979daa6ce5b8499588"
I20260806 06:36:14.368008 9352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d19d529bd2b48b5b2119a5d6fafeb71" candidate_uuid: "87816cb844af4ed5a225b36dfa101298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9e048616d347d7aafc8191fc2e9838"
I20260806 06:36:14.368050 9153 raft_consensus.cc:3037] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:14.368101 9352 raft_consensus.cc:3037] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:14.368820 9352 raft_consensus.cc:2454] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87816cb844af4ed5a225b36dfa101298 in term 1.
I20260806 06:36:14.368831 9153 raft_consensus.cc:2454] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87816cb844af4ed5a225b36dfa101298 in term 1.
I20260806 06:36:14.369012 8841 leader_election.cc:302] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c9e048616d347d7aafc8191fc2e9838, 87816cb844af4ed5a225b36dfa101298; no voters:
I20260806 06:36:14.369107 9469 raft_consensus.cc:2781] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:14.369246 9469 raft_consensus.cc:686] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 1 LEADER]: Becoming Leader. State: Replica: 87816cb844af4ed5a225b36dfa101298, State: Running, Role: LEADER
I20260806 06:36:14.369349 9469 consensus_queue.cc:237] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } }
I20260806 06:36:14.370253 8707 catalog_manager.cc:5495] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87816cb844af4ed5a225b36dfa101298 (127.29.142.129). New cstate: current_term: 1 leader_uuid: "87816cb844af4ed5a225b36dfa101298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } health_report { overall_health: UNKNOWN } } }
I20260806 06:36:14.404242 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.401598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53668 (local address 127.29.142.129:46751)
0806 06:36:14.401664 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.401666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.401676 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:36:14.401740 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.401741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.401762 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.401831 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:36:14.401970 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.402457 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.403439 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.403648 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.403717 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.403878 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:36:14.403989 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:36:14.404057 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.404074 (+ 17us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.404095 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:36:14.406308 9153 raft_consensus.cc:1261] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Refusing update from remote peer 87816cb844af4ed5a225b36dfa101298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:14.406318 9352 raft_consensus.cc:1261] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Refusing update from remote peer 87816cb844af4ed5a225b36dfa101298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:14.406692 9469 consensus_queue.cc:1035] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:14.406883 9473 consensus_queue.cc:1035] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:14.409968 9479 mvcc.cc:205] Tried to move back new op lower bound from 7315448522365464576 to 7315448522217627648. Current Snapshot: MvccSnapshot[applied={T|T < 7315448522365464576}]
I20260806 06:36:14.410136 9480 mvcc.cc:205] Tried to move back new op lower bound from 7315448522365464576 to 7315448522217627648. Current Snapshot: MvccSnapshot[applied={T|T < 7315448522365464576}]
I20260806 06:36:14.411878 8708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:14.411985 8708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:14.412810 8708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5: Sending DeleteTablet for 3 replicas of tablet 4d19d529bd2b48b5b2119a5d6fafeb71
I20260806 06:36:14.413141 9312 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d19d529bd2b48b5b2119a5d6fafeb71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:14 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:34386
I20260806 06:36:14.413168 9120 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d19d529bd2b48b5b2119a5d6fafeb71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:14 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:45408
I20260806 06:36:14.413411 9491 tablet_replica.cc:330] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838: stopping tablet replica
I20260806 06:36:14.413403 8708 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 06:36:14.413518 9491 raft_consensus.cc:2229] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 06:36:14.413555 8708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:36:14.413924 9491 raft_consensus.cc:2258] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:14.413995 8904 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d19d529bd2b48b5b2119a5d6fafeb71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:14 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:53658
I20260806 06:36:14.414354 9493 tablet_replica.cc:330] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298: stopping tablet replica
I20260806 06:36:14.414516 9493 raft_consensus.cc:2229] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:36:14.414588 9492 tablet_replica.cc:330] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588: stopping tablet replica
I20260806 06:36:14.414752 9493 raft_consensus.cc:2258] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:14.414753 9492 raft_consensus.cc:2229] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:14.414886 9492 raft_consensus.cc:2258] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:14.415300 9493 ts_tablet_manager.cc:1905] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:14.415560 9492 ts_tablet_manager.cc:1905] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:14.415822 9312 tablet_service.cc:1467] Processing CreateTablet for tablet 852de640f64b4a75a01001bdc0ba219b (DEFAULT_TABLE table=test-table [id=36d87a144fe647abab03d5b78dbfccf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:14.415951 9312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 852de640f64b4a75a01001bdc0ba219b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:14.417349 9493 ts_tablet_manager.cc:1918] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:14.417382 9456 tablet_bootstrap.cc:492] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838: Bootstrap starting.
I20260806 06:36:14.417388 9492 ts_tablet_manager.cc:1918] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:14.417455 9492 log.cc:1196] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/4d19d529bd2b48b5b2119a5d6fafeb71
I20260806 06:36:14.417569 9493 log.cc:1196] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/wal/wals/4d19d529bd2b48b5b2119a5d6fafeb71
I20260806 06:36:14.417789 9492 ts_tablet_manager.cc:1939] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 98116dbe143a48979daa6ce5b8499588: Deleting consensus metadata
I20260806 06:36:14.417840 9456 tablet_bootstrap.cc:654] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:14.418012 9493 ts_tablet_manager.cc:1939] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 87816cb844af4ed5a225b36dfa101298: Deleting consensus metadata
I20260806 06:36:14.418498 8688 catalog_manager.cc:4841] TS 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751): tablet 4d19d529bd2b48b5b2119a5d6fafeb71 (table test-table [id=b2ad39c8541e4edaaccf6876cf4070dd]) successfully deleted
I20260806 06:36:14.418570 8904 tablet_service.cc:1467] Processing CreateTablet for tablet 852de640f64b4a75a01001bdc0ba219b (DEFAULT_TABLE table=test-table [id=36d87a144fe647abab03d5b78dbfccf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:14.418634 8688 catalog_manager.cc:4841] TS 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911): tablet 4d19d529bd2b48b5b2119a5d6fafeb71 (table test-table [id=b2ad39c8541e4edaaccf6876cf4070dd]) successfully deleted
I20260806 06:36:14.418705 8904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 852de640f64b4a75a01001bdc0ba219b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:14.418834 9120 tablet_service.cc:1467] Processing CreateTablet for tablet 852de640f64b4a75a01001bdc0ba219b (DEFAULT_TABLE table=test-table [id=36d87a144fe647abab03d5b78dbfccf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:14.418941 9120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 852de640f64b4a75a01001bdc0ba219b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:14.419732 9457 tablet_bootstrap.cc:492] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588: Bootstrap starting.
I20260806 06:36:14.420156 9491 ts_tablet_manager.cc:1905] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:14.420213 9457 tablet_bootstrap.cc:654] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:14.420574 9459 tablet_bootstrap.cc:492] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298: Bootstrap starting.
I20260806 06:36:14.420992 9459 tablet_bootstrap.cc:654] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:14.421669 9459 tablet_bootstrap.cc:492] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298: No bootstrap required, opened a new log
I20260806 06:36:14.421720 9459 ts_tablet_manager.cc:1397] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:14.421864 9459 raft_consensus.cc:348] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.421902 9491 ts_tablet_manager.cc:1918] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:14.421936 9459 raft_consensus.cc:374] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:14.421958 9459 raft_consensus.cc:729] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87816cb844af4ed5a225b36dfa101298, State: Initialized, Role: FOLLOWER
I20260806 06:36:14.421979 9491 log.cc:1196] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/4d19d529bd2b48b5b2119a5d6fafeb71
I20260806 06:36:14.422006 9457 tablet_bootstrap.cc:492] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588: No bootstrap required, opened a new log
I20260806 06:36:14.422003 9459 consensus_queue.cc:260] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.422051 9457 ts_tablet_manager.cc:1397] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:36:14.422101 9459 ts_tablet_manager.cc:1428] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:14.422217 9457 raft_consensus.cc:348] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.422323 9457 raft_consensus.cc:374] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:14.422371 9457 raft_consensus.cc:729] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98116dbe143a48979daa6ce5b8499588, State: Initialized, Role: FOLLOWER
I20260806 06:36:14.422416 9491 ts_tablet_manager.cc:1939] T 4d19d529bd2b48b5b2119a5d6fafeb71 P 7c9e048616d347d7aafc8191fc2e9838: Deleting consensus metadata
I20260806 06:36:14.422422 9457 consensus_queue.cc:260] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.422539 9457 ts_tablet_manager.cc:1428] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:14.422865 8690 catalog_manager.cc:4841] TS 7c9e048616d347d7aafc8191fc2e9838 (127.29.142.131:41265): tablet 4d19d529bd2b48b5b2119a5d6fafeb71 (table test-table [id=b2ad39c8541e4edaaccf6876cf4070dd]) successfully deleted
I20260806 06:36:14.423350 9456 tablet_bootstrap.cc:492] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838: No bootstrap required, opened a new log
I20260806 06:36:14.423409 9456 ts_tablet_manager.cc:1397] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260806 06:36:14.423560 9456 raft_consensus.cc:348] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.423637 9456 raft_consensus.cc:374] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:14.423668 9456 raft_consensus.cc:729] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9e048616d347d7aafc8191fc2e9838, State: Initialized, Role: FOLLOWER
I20260806 06:36:14.423789 9456 consensus_queue.cc:260] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.423965 9456 ts_tablet_manager.cc:1428] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:14.604714 9467 raft_consensus.cc:482] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:36:14.604852 9467 raft_consensus.cc:504] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.605180 9467 leader_election.cc:288] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751), 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911)
I20260806 06:36:14.609552 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.605514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:35721 (local address 127.29.142.129:46751)
0806 06:36:14.605624 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.605627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.605644 (+ 17us) server_negotiation.cc:407] Connection header received
0806 06:36:14.605705 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.605706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.605721 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.605816 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:14.606056 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.606952 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.608556 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.609125 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.609220 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.609232 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.609425 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.609472 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 06:36:14.609552 9455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.605666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:32955 (local address 127.29.142.130:42911)
0806 06:36:14.605739 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.605741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.605752 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:14.605801 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.605802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.605813 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.605873 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:14.606050 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.606926 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.608556 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.609125 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.609230 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.609240 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.609425 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.609472 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:36:14.609570 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.605404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:46751 (local address 127.29.142.131:35721)
0806 06:36:14.605472 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.605486 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.605542 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.605834 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.605835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.605837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:14.605955 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.605959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.607072 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.607073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.608418 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.608422 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.609230 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.609232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.609338 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.609340 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.609406 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.609466 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":108}
I20260806 06:36:14.609586 9499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.605536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:42911 (local address 127.29.142.131:32955)
0806 06:36:14.605682 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.605693 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.605744 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.605881 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.605882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.605884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:14.605971 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.605973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.607072 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.607073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.608418 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.608422 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.609253 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.609254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.609326 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.609340 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.609406 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.609466 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":16,"thread_start_us":86,"threads_started":1}
I20260806 06:36:14.609925 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "852de640f64b4a75a01001bdc0ba219b" candidate_uuid: "7c9e048616d347d7aafc8191fc2e9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98116dbe143a48979daa6ce5b8499588" is_pre_election: true
I20260806 06:36:14.609964 8936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "852de640f64b4a75a01001bdc0ba219b" candidate_uuid: "7c9e048616d347d7aafc8191fc2e9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87816cb844af4ed5a225b36dfa101298" is_pre_election: true
I20260806 06:36:14.610047 9153 raft_consensus.cc:2454] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9e048616d347d7aafc8191fc2e9838 in term 0.
I20260806 06:36:14.610109 8936 raft_consensus.cc:2454] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9e048616d347d7aafc8191fc2e9838 in term 0.
I20260806 06:36:14.610252 9264 leader_election.cc:302] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c9e048616d347d7aafc8191fc2e9838, 98116dbe143a48979daa6ce5b8499588; no voters:
I20260806 06:36:14.610402 9467 raft_consensus.cc:2781] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:36:14.610472 9467 raft_consensus.cc:482] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:36:14.610505 9467 raft_consensus.cc:3037] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:14.611325 9467 raft_consensus.cc:504] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.611485 9467 leader_election.cc:288] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [CANDIDATE]: Term 1 election: Requested vote from peers 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751), 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911)
I20260806 06:36:14.611680 8936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "852de640f64b4a75a01001bdc0ba219b" candidate_uuid: "7c9e048616d347d7aafc8191fc2e9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87816cb844af4ed5a225b36dfa101298"
I20260806 06:36:14.611711 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "852de640f64b4a75a01001bdc0ba219b" candidate_uuid: "7c9e048616d347d7aafc8191fc2e9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98116dbe143a48979daa6ce5b8499588"
I20260806 06:36:14.611783 8936 raft_consensus.cc:3037] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:14.611809 9153 raft_consensus.cc:3037] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:14.612565 8936 raft_consensus.cc:2454] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9e048616d347d7aafc8191fc2e9838 in term 1.
I20260806 06:36:14.612607 9153 raft_consensus.cc:2454] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9e048616d347d7aafc8191fc2e9838 in term 1.
I20260806 06:36:14.612820 9264 leader_election.cc:302] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c9e048616d347d7aafc8191fc2e9838, 98116dbe143a48979daa6ce5b8499588; no voters:
I20260806 06:36:14.612924 9467 raft_consensus.cc:2781] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:14.613133 9467 raft_consensus.cc:686] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 1 LEADER]: Becoming Leader. State: Replica: 7c9e048616d347d7aafc8191fc2e9838, State: Running, Role: LEADER
I20260806 06:36:14.613220 9467 consensus_queue.cc:237] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.614022 8707 catalog_manager.cc:5495] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c9e048616d347d7aafc8191fc2e9838 (127.29.142.131). New cstate: current_term: 1 leader_uuid: "7c9e048616d347d7aafc8191fc2e9838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } health_report { overall_health: HEALTHY } } }
Aug 06 06:36:14 dist-test-slave-3kk6 krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998173, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.190@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
W20260806 06:36:14.670804 9418 tablet.cc:2369] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:36:14.670967 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.663593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45898 (local address 127.29.142.190:37211)
0806 06:36:14.663707 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.663709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.663786 (+ 77us) server_negotiation.cc:407] Connection header received
0806 06:36:14.663854 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.663855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.663866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.663911 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:14.664397 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.664891 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.665594 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.665776 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.665836 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.668241 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:14.668242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:14.668243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:14.668251 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:14.669702 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:14.670118 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:14.670119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:14.670120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:14.670180 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:14.670372 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:14.670373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:14.670373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:14.670486 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:14.670578 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.670758 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.670851 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 06:36:14.671037 9515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.663523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37211 (local address 127.0.0.1:45898)
0806 06:36:14.663693 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.663705 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.663791 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.663927 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.663928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.664265 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:14.664324 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.664326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.664968 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.664969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.665515 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.665517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.665847 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.665848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.665895 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.666507 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:14.666512 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:14.668140 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:14.669807 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:14.669817 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:14.669820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:14.670039 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:14.670245 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:14.670246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:14.670246 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:14.670309 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:14.670560 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:14.670569 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:14.670618 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.670732 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.670911 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":56,"thread_start_us":77,"threads_started":1}
I20260806 06:36:14.673527 8707 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:45898:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:36:14.673830 8707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:36:14.675613 9312 tablet_service.cc:1467] Processing CreateTablet for tablet de3c01a5fe4842a3955da1527645644b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=346bd414917547ecb03cfb1807384bc6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:36:14.675654 8904 tablet_service.cc:1467] Processing CreateTablet for tablet de3c01a5fe4842a3955da1527645644b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=346bd414917547ecb03cfb1807384bc6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:36:14.675799 9312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de3c01a5fe4842a3955da1527645644b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:14.675848 8904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de3c01a5fe4842a3955da1527645644b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:14.677018 9456 tablet_bootstrap.cc:492] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838: Bootstrap starting.
I20260806 06:36:14.677050 9459 tablet_bootstrap.cc:492] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298: Bootstrap starting.
I20260806 06:36:14.677140 9120 tablet_service.cc:1467] Processing CreateTablet for tablet de3c01a5fe4842a3955da1527645644b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=346bd414917547ecb03cfb1807384bc6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:36:14.677309 9120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de3c01a5fe4842a3955da1527645644b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:14.677448 9459 tablet_bootstrap.cc:654] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:14.677474 9456 tablet_bootstrap.cc:654] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:14.678122 9459 tablet_bootstrap.cc:492] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298: No bootstrap required, opened a new log
I20260806 06:36:14.678164 9456 tablet_bootstrap.cc:492] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838: No bootstrap required, opened a new log
I20260806 06:36:14.678215 9456 ts_tablet_manager.cc:1397] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:14.678205 9459 ts_tablet_manager.cc:1397] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:14.678344 9459 raft_consensus.cc:348] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.678366 9456 raft_consensus.cc:348] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.678440 9456 raft_consensus.cc:374] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:14.678442 9459 raft_consensus.cc:374] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:14.678460 9456 raft_consensus.cc:729] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9e048616d347d7aafc8191fc2e9838, State: Initialized, Role: FOLLOWER
I20260806 06:36:14.678471 9459 raft_consensus.cc:729] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87816cb844af4ed5a225b36dfa101298, State: Initialized, Role: FOLLOWER
I20260806 06:36:14.678509 9456 consensus_queue.cc:260] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.678519 9459 consensus_queue.cc:260] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.678629 9456 ts_tablet_manager.cc:1428] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:14.678632 9459 ts_tablet_manager.cc:1428] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:14.678725 9467 tablet_replica.cc:439] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } }
I20260806 06:36:14.678828 9467 tablet_replica.cc:442] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:36:14.679033 9469 tablet_replica.cc:439] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } }
I20260806 06:36:14.679113 9469 tablet_replica.cc:442] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:36:14.679230 9457 tablet_bootstrap.cc:492] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: Bootstrap starting.
I20260806 06:36:14.679662 9457 tablet_bootstrap.cc:654] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:14.680347 9457 tablet_bootstrap.cc:492] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: No bootstrap required, opened a new log
I20260806 06:36:14.680390 9457 ts_tablet_manager.cc:1397] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:14.680544 9457 raft_consensus.cc:348] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.680609 9457 raft_consensus.cc:374] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:14.680625 9457 raft_consensus.cc:729] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98116dbe143a48979daa6ce5b8499588, State: Initialized, Role: FOLLOWER
I20260806 06:36:14.680693 9457 consensus_queue.cc:260] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.680809 9457 ts_tablet_manager.cc:1428] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:14.680879 9468 tablet_replica.cc:439] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } }
I20260806 06:36:14.681005 9468 tablet_replica.cc:442] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:36:14.742064 9468 raft_consensus.cc:482] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:36:14.742182 9468 raft_consensus.cc:504] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.742576 9468 leader_election.cc:288] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751), 7c9e048616d347d7aafc8191fc2e9838 (127.29.142.131:41265)
I20260806 06:36:14.745997 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.742892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:47821 (local address 127.29.142.131:41265)
0806 06:36:14.742950 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.742954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.743050 (+ 96us) server_negotiation.cc:407] Connection header received
0806 06:36:14.743131 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.743132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.743147 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.743212 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:14.743408 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.743958 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.745080 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.745579 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.745659 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.745669 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.745875 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.745910 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:36:14.746023 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.742921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:47513 (local address 127.29.142.129:46751)
0806 06:36:14.742996 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.742998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.743074 (+ 76us) server_negotiation.cc:407] Connection header received
0806 06:36:14.743177 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.743178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.743193 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.743252 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:14.743409 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.744012 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.745197 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.745669 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.745753 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.745762 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.745909 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.745953 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:36:14.746065 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.742828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:41265 (local address 127.29.142.130:47821)
0806 06:36:14.742969 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.742981 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.743069 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.743224 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.743225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.743228 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:14.743334 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.743336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.744072 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.744073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.744975 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.744978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.745660 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.745661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.745788 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.745793 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.745866 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.745910 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":41,"thread_start_us":73,"threads_started":1}
I20260806 06:36:14.746073 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.742863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:46751 (local address 127.29.142.130:47513)
0806 06:36:14.742991 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.743003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.743093 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.743257 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.743258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.743260 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:14.743339 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.743341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.744093 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.744094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.745113 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.745116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.745753 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.745754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.745828 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.745832 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.745904 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.745947 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":16,"thread_start_us":64,"threads_started":1}
I20260806 06:36:14.746430 8936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de3c01a5fe4842a3955da1527645644b" candidate_uuid: "98116dbe143a48979daa6ce5b8499588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87816cb844af4ed5a225b36dfa101298" is_pre_election: true
I20260806 06:36:14.746488 9352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de3c01a5fe4842a3955da1527645644b" candidate_uuid: "98116dbe143a48979daa6ce5b8499588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9e048616d347d7aafc8191fc2e9838" is_pre_election: true
I20260806 06:36:14.746539 8936 raft_consensus.cc:2454] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98116dbe143a48979daa6ce5b8499588 in term 0.
I20260806 06:36:14.746588 9352 raft_consensus.cc:2454] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98116dbe143a48979daa6ce5b8499588 in term 0.
I20260806 06:36:14.746851 9045 leader_election.cc:302] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c9e048616d347d7aafc8191fc2e9838, 98116dbe143a48979daa6ce5b8499588; no voters:
I20260806 06:36:14.747017 9468 raft_consensus.cc:2781] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:36:14.747072 9468 raft_consensus.cc:482] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:36:14.747105 9468 raft_consensus.cc:3037] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:14.747924 9468 raft_consensus.cc:504] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.748081 9468 leader_election.cc:288] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [CANDIDATE]: Term 1 election: Requested vote from peers 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751), 7c9e048616d347d7aafc8191fc2e9838 (127.29.142.131:41265)
I20260806 06:36:14.748251 8936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de3c01a5fe4842a3955da1527645644b" candidate_uuid: "98116dbe143a48979daa6ce5b8499588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87816cb844af4ed5a225b36dfa101298"
I20260806 06:36:14.748261 9352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de3c01a5fe4842a3955da1527645644b" candidate_uuid: "98116dbe143a48979daa6ce5b8499588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9e048616d347d7aafc8191fc2e9838"
I20260806 06:36:14.748329 8936 raft_consensus.cc:3037] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:14.748344 9352 raft_consensus.cc:3037] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:14.748963 8936 raft_consensus.cc:2454] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98116dbe143a48979daa6ce5b8499588 in term 1.
I20260806 06:36:14.748974 9352 raft_consensus.cc:2454] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98116dbe143a48979daa6ce5b8499588 in term 1.
I20260806 06:36:14.749159 9044 leader_election.cc:302] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87816cb844af4ed5a225b36dfa101298, 98116dbe143a48979daa6ce5b8499588; no voters:
I20260806 06:36:14.749294 9468 raft_consensus.cc:2781] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:14.749431 9468 raft_consensus.cc:686] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [term 1 LEADER]: Becoming Leader. State: Replica: 98116dbe143a48979daa6ce5b8499588, State: Running, Role: LEADER
I20260806 06:36:14.749516 9468 consensus_queue.cc:237] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.749877 9526 tablet_replica.cc:439] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: TxnStatusTablet state changed. Reason: New leader 98116dbe143a48979daa6ce5b8499588. Latest consensus state: current_term: 1 leader_uuid: "98116dbe143a48979daa6ce5b8499588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } }
I20260806 06:36:14.749993 9526 tablet_replica.cc:442] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:36:14.750191 9529 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:36:14.750308 9529 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:36:14.750277 8707 catalog_manager.cc:5495] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98116dbe143a48979daa6ce5b8499588 (127.29.142.130). New cstate: current_term: 1 leader_uuid: "98116dbe143a48979daa6ce5b8499588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } health_report { overall_health: UNKNOWN } } }
I20260806 06:36:14.756273 9455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.753535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45414 (local address 127.29.142.130:42911)
0806 06:36:14.753576 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.753578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.753587 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:36:14.753661 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.753661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.753676 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.753735 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:36:14.753900 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.754360 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.755494 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.755668 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.755723 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.755961 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:36:14.756054 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:36:14.756125 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.756203 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.756220 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:36:14.756343 9515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.753458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:42911 (local address 127.0.0.1:45414)
0806 06:36:14.753529 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.753543 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.753597 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.753761 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.753762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.753763 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:36:14.753819 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.753830 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.754438 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.754439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.755412 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.755416 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.755742 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.755743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.755809 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.755834 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:36:14.756130 (+ 296us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:36:14.756139 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.756201 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.756256 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 06:36:14.757597 8936 raft_consensus.cc:1261] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Refusing update from remote peer 98116dbe143a48979daa6ce5b8499588: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:14.757607 9352 raft_consensus.cc:1261] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Refusing update from remote peer 98116dbe143a48979daa6ce5b8499588: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:14.757910 9468 consensus_queue.cc:1035] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:14.758044 9526 consensus_queue.cc:1035] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:14.758841 9469 tablet_replica.cc:439] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298: TxnStatusTablet state changed. Reason: New leader 98116dbe143a48979daa6ce5b8499588. Latest consensus state: current_term: 1 leader_uuid: "98116dbe143a48979daa6ce5b8499588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } }
I20260806 06:36:14.758936 9469 tablet_replica.cc:442] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:36:14.759114 9467 tablet_replica.cc:439] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838: TxnStatusTablet state changed. Reason: New leader 98116dbe143a48979daa6ce5b8499588. Latest consensus state: current_term: 1 leader_uuid: "98116dbe143a48979daa6ce5b8499588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } }
I20260806 06:36:14.759207 9467 tablet_replica.cc:442] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:36:14.759601 9468 tablet_replica.cc:439] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98116dbe143a48979daa6ce5b8499588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } }
I20260806 06:36:14.759685 9468 tablet_replica.cc:442] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:36:14.760142 9467 tablet_replica.cc:439] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98116dbe143a48979daa6ce5b8499588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } }
I20260806 06:36:14.760226 9467 tablet_replica.cc:442] T de3c01a5fe4842a3955da1527645644b P 7c9e048616d347d7aafc8191fc2e9838: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:36:14.760651 9468 tablet_replica.cc:439] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98116dbe143a48979daa6ce5b8499588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } }
I20260806 06:36:14.760748 9468 tablet_replica.cc:442] T de3c01a5fe4842a3955da1527645644b P 98116dbe143a48979daa6ce5b8499588: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:36:14.761261 9478 mvcc.cc:205] Tried to move back new op lower bound from 7315448523804504064 to 7315448523774619648. Current Snapshot: MvccSnapshot[applied={T|T < 7315448523804504064}]
I20260806 06:36:14.761351 9469 tablet_replica.cc:439] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98116dbe143a48979daa6ce5b8499588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } } }
I20260806 06:36:14.761447 9469 tablet_replica.cc:442] T de3c01a5fe4842a3955da1527645644b P 87816cb844af4ed5a225b36dfa101298: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:36:14.764338 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.761494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34394 (local address 127.29.142.131:41265)
0806 06:36:14.761533 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.761535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.761542 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:36:14.761578 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.761579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.761600 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.761680 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:36:14.761832 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.762567 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.763454 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.763752 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.763831 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.763989 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:36:14.764117 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:36:14.764184 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.764247 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.764266 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:36:14.769526 9455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.766923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:59243 (local address 127.29.142.130:42911)
0806 06:36:14.766998 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.766999 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.767067 (+ 68us) server_negotiation.cc:407] Connection header received
0806 06:36:14.767143 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.767143 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.767152 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.767193 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:36:14.767331 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.767783 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.768899 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.769077 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.769132 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.769330 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:36:14.769357 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:36:14.769396 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.769452 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.769469 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:36:14.769577 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.766837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:42911 (local address 127.29.142.131:59243)
0806 06:36:14.767008 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.767020 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.767082 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.767205 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.767206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.767207 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:36:14.767272 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.767274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.767864 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.767865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.768798 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.768800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.769157 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.769158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.769226 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.769244 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:36:14.769398 (+ 154us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:36:14.769399 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.769448 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.769490 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260806 06:36:14.772390 8936 raft_consensus.cc:1261] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Refusing update from remote peer 7c9e048616d347d7aafc8191fc2e9838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:14.772390 9153 raft_consensus.cc:1261] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Refusing update from remote peer 7c9e048616d347d7aafc8191fc2e9838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:14.772967 9500 consensus_queue.cc:1035] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:14.773140 9467 consensus_queue.cc:1035] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:14.782697 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.779691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:45763 (local address 127.29.142.131:41265)
0806 06:36:14.779730 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.779732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.779741 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:36:14.779843 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.779844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.779854 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.779921 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:36:14.780050 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.780815 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.781828 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.782138 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.782206 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.782379 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:36:14.782415 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:36:14.782471 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.782571 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.782592 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:36:14.782747 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.779503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:41265 (local address 127.29.142.130:45763)
0806 06:36:14.779672 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:14.779686 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:14.779738 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:14.779923 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:14.779924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:14.779925 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:36:14.779996 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.779997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.780903 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.780903 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.781746 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:14.781749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.782245 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.782246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:14.782291 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.782306 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:36:14.782504 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:36:14.782507 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:36:14.782563 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:36:14.782604 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260806 06:36:14.791726 8708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:14.791858 8708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:14.792531 8708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5: Sending DeleteTablet for 3 replicas of tablet 852de640f64b4a75a01001bdc0ba219b
I20260806 06:36:14.792793 9312 tablet_service.cc:1514] Processing DeleteTablet for tablet 852de640f64b4a75a01001bdc0ba219b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:14 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:34386
I20260806 06:36:14.792999 8904 tablet_service.cc:1514] Processing DeleteTablet for tablet 852de640f64b4a75a01001bdc0ba219b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:14 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:53658
I20260806 06:36:14.793114 9491 tablet_replica.cc:330] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838: stopping tablet replica
I20260806 06:36:14.793192 9491 raft_consensus.cc:2229] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:36:14.793289 9491 raft_consensus.cc:2258] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:14.793385 9493 tablet_replica.cc:330] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298: stopping tablet replica
I20260806 06:36:14.793450 9493 raft_consensus.cc:2229] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:14.793491 9493 raft_consensus.cc:2258] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:14.794168 9493 ts_tablet_manager.cc:1905] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:14.794745 9120 tablet_service.cc:1514] Processing DeleteTablet for tablet 852de640f64b4a75a01001bdc0ba219b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:14 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:45408
I20260806 06:36:14.794867 9492 tablet_replica.cc:330] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588: stopping tablet replica
I20260806 06:36:14.794931 9492 raft_consensus.cc:2229] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:14.794977 9492 raft_consensus.cc:2258] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:14.796005 9493 ts_tablet_manager.cc:1918] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:36:14.796061 9493 log.cc:1196] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/wal/wals/852de640f64b4a75a01001bdc0ba219b
I20260806 06:36:14.796295 9493 ts_tablet_manager.cc:1939] T 852de640f64b4a75a01001bdc0ba219b P 87816cb844af4ed5a225b36dfa101298: Deleting consensus metadata
I20260806 06:36:14.797453 8688 catalog_manager.cc:4841] TS 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751): tablet 852de640f64b4a75a01001bdc0ba219b (table test-table [id=36d87a144fe647abab03d5b78dbfccf9]) successfully deleted
I20260806 06:36:14.797549 9492 ts_tablet_manager.cc:1905] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:14.798990 9491 ts_tablet_manager.cc:1905] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:14.800643 9491 ts_tablet_manager.cc:1918] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:36:14.800704 9491 log.cc:1196] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/852de640f64b4a75a01001bdc0ba219b
I20260806 06:36:14.800935 9491 ts_tablet_manager.cc:1939] T 852de640f64b4a75a01001bdc0ba219b P 7c9e048616d347d7aafc8191fc2e9838: Deleting consensus metadata
I20260806 06:36:14.801357 8690 catalog_manager.cc:4841] TS 7c9e048616d347d7aafc8191fc2e9838 (127.29.142.131:41265): tablet 852de640f64b4a75a01001bdc0ba219b (table test-table [id=36d87a144fe647abab03d5b78dbfccf9]) successfully deleted
I20260806 06:36:14.803122 9492 ts_tablet_manager.cc:1918] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:36:14.803191 9492 log.cc:1196] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/852de640f64b4a75a01001bdc0ba219b
I20260806 06:36:14.803454 9492 ts_tablet_manager.cc:1939] T 852de640f64b4a75a01001bdc0ba219b P 98116dbe143a48979daa6ce5b8499588: Deleting consensus metadata
I20260806 06:36:14.803809 8688 catalog_manager.cc:4841] TS 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911): tablet 852de640f64b4a75a01001bdc0ba219b (table test-table [id=36d87a144fe647abab03d5b78dbfccf9]) successfully deleted
Aug 06 06:36:14 dist-test-slave-3kk6 krb5kdc[8645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:36:14 dist-test-slave-3kk6 krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:14 dist-test-slave-3kk6 krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:36:14.831503 8786 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:14.822436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45912 (local address 127.29.142.190:37211)
0806 06:36:14.822496 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.822499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.822638 (+ 139us) server_negotiation.cc:407] Connection header received
0806 06:36:14.822815 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.822817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.822829 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.822888 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:14.823666 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.824471 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.825611 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.825939 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.826017 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.831434 (+ 5417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45912: BlockingRecv error: recv got EOF from 127.0.0.1:45912 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 06:36:14 dist-test-slave-3kk6 krb5kdc[8645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM
I20260806 06:36:14.842478 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:14.833018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45924 (local address 127.29.142.190:37211)
0806 06:36:14.833067 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:14.833069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:14.833248 (+ 179us) server_negotiation.cc:407] Connection header received
0806 06:36:14.833361 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:14.833362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:14.833374 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:14.833407 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:14.834105 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.834990 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.836059 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:14.836394 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:14.836472 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:14.839408 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:14.839410 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:14.839411 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:14.839419 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:14.841173 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:14.841586 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:14.841588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:14.841588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:14.841655 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:14.841879 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:14.841880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:14.841881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:14.842027 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:14.842131 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:14.842289 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 06:36:14.842378 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:36:14.844679 8708 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:36:14.844888 8708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:36:14.846354 9312 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3860a57e014645b6219568f1d430dd (DEFAULT_TABLE table=test-table [id=b0414e1b91b946b59b4ac70b819a203f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:14.846418 8904 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3860a57e014645b6219568f1d430dd (DEFAULT_TABLE table=test-table [id=b0414e1b91b946b59b4ac70b819a203f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:14.846514 9312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3860a57e014645b6219568f1d430dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:14.846562 8904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3860a57e014645b6219568f1d430dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:14.848097 9120 tablet_service.cc:1467] Processing CreateTablet for tablet 2b3860a57e014645b6219568f1d430dd (DEFAULT_TABLE table=test-table [id=b0414e1b91b946b59b4ac70b819a203f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:14.848132 9456 tablet_bootstrap.cc:492] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838: Bootstrap starting.
I20260806 06:36:14.848238 9120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b3860a57e014645b6219568f1d430dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:14.848594 9456 tablet_bootstrap.cc:654] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:14.849284 9456 tablet_bootstrap.cc:492] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838: No bootstrap required, opened a new log
I20260806 06:36:14.849359 9456 ts_tablet_manager.cc:1397] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:14.849548 9459 tablet_bootstrap.cc:492] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298: Bootstrap starting.
I20260806 06:36:14.849519 9456 raft_consensus.cc:348] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.849602 9456 raft_consensus.cc:374] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:14.849625 9456 raft_consensus.cc:729] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9e048616d347d7aafc8191fc2e9838, State: Initialized, Role: FOLLOWER
I20260806 06:36:14.849679 9456 consensus_queue.cc:260] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.849773 9456 ts_tablet_manager.cc:1428] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:14.849881 9459 tablet_bootstrap.cc:654] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:14.850152 9457 tablet_bootstrap.cc:492] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588: Bootstrap starting.
I20260806 06:36:14.850463 9457 tablet_bootstrap.cc:654] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:14.851226 9459 tablet_bootstrap.cc:492] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298: No bootstrap required, opened a new log
I20260806 06:36:14.851280 9459 ts_tablet_manager.cc:1397] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:36:14.851389 9459 raft_consensus.cc:348] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.851461 9459 raft_consensus.cc:374] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:14.851481 9459 raft_consensus.cc:729] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87816cb844af4ed5a225b36dfa101298, State: Initialized, Role: FOLLOWER
I20260806 06:36:14.851519 9459 consensus_queue.cc:260] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.851603 9459 ts_tablet_manager.cc:1428] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:36:14.852684 9457 tablet_bootstrap.cc:492] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588: No bootstrap required, opened a new log
I20260806 06:36:14.852741 9457 ts_tablet_manager.cc:1397] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260806 06:36:14.852891 9457 raft_consensus.cc:348] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.852962 9457 raft_consensus.cc:374] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:14.852983 9457 raft_consensus.cc:729] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98116dbe143a48979daa6ce5b8499588, State: Initialized, Role: FOLLOWER
I20260806 06:36:14.853032 9457 consensus_queue.cc:260] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.853114 9457 ts_tablet_manager.cc:1428] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:14.996341 9545 raft_consensus.cc:482] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:36:14.996479 9545 raft_consensus.cc:504] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.996728 9545 leader_election.cc:288] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751), 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911)
I20260806 06:36:14.997069 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3860a57e014645b6219568f1d430dd" candidate_uuid: "7c9e048616d347d7aafc8191fc2e9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98116dbe143a48979daa6ce5b8499588" is_pre_election: true
I20260806 06:36:14.997069 8936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3860a57e014645b6219568f1d430dd" candidate_uuid: "7c9e048616d347d7aafc8191fc2e9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87816cb844af4ed5a225b36dfa101298" is_pre_election: true
I20260806 06:36:14.997195 9153 raft_consensus.cc:2454] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9e048616d347d7aafc8191fc2e9838 in term 0.
I20260806 06:36:14.997210 8936 raft_consensus.cc:2454] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9e048616d347d7aafc8191fc2e9838 in term 0.
I20260806 06:36:14.997397 9264 leader_election.cc:302] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c9e048616d347d7aafc8191fc2e9838, 98116dbe143a48979daa6ce5b8499588; no voters:
I20260806 06:36:14.997499 9545 raft_consensus.cc:2781] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:36:14.997556 9545 raft_consensus.cc:482] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:36:14.997589 9545 raft_consensus.cc:3037] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:14.998381 9545 raft_consensus.cc:504] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:14.998536 9545 leader_election.cc:288] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [CANDIDATE]: Term 1 election: Requested vote from peers 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751), 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911)
I20260806 06:36:14.998800 8936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3860a57e014645b6219568f1d430dd" candidate_uuid: "7c9e048616d347d7aafc8191fc2e9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87816cb844af4ed5a225b36dfa101298"
I20260806 06:36:14.998875 8936 raft_consensus.cc:3037] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:14.998821 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b3860a57e014645b6219568f1d430dd" candidate_uuid: "7c9e048616d347d7aafc8191fc2e9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98116dbe143a48979daa6ce5b8499588"
I20260806 06:36:14.998925 9153 raft_consensus.cc:3037] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:14.999549 9153 raft_consensus.cc:2454] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9e048616d347d7aafc8191fc2e9838 in term 1.
I20260806 06:36:14.999558 8936 raft_consensus.cc:2454] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9e048616d347d7aafc8191fc2e9838 in term 1.
I20260806 06:36:14.999752 9264 leader_election.cc:302] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c9e048616d347d7aafc8191fc2e9838, 98116dbe143a48979daa6ce5b8499588; no voters:
I20260806 06:36:14.999859 9545 raft_consensus.cc:2781] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:14.999919 9545 raft_consensus.cc:686] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 1 LEADER]: Becoming Leader. State: Replica: 7c9e048616d347d7aafc8191fc2e9838, State: Running, Role: LEADER
I20260806 06:36:14.999991 9545 consensus_queue.cc:237] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:15.000612 8707 catalog_manager.cc:5495] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c9e048616d347d7aafc8191fc2e9838 (127.29.142.131). New cstate: current_term: 1 leader_uuid: "7c9e048616d347d7aafc8191fc2e9838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } health_report { overall_health: HEALTHY } } }
I20260806 06:36:15.037362 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:15.035331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34408 (local address 127.29.142.131:41265)
0806 06:36:15.035384 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:15.035386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:15.035396 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:36:15.035423 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:15.035424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:15.035433 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:15.035485 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:36:15.035600 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:15.036071 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:15.036819 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:15.037005 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:15.037057 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:15.037170 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:36:15.037195 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:36:15.037228 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:15.037277 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:36:15.037297 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:36:15.038300 8936 raft_consensus.cc:1261] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Refusing update from remote peer 7c9e048616d347d7aafc8191fc2e9838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:15.038307 9153 raft_consensus.cc:1261] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Refusing update from remote peer 7c9e048616d347d7aafc8191fc2e9838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:15.038568 9500 consensus_queue.cc:1035] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:15.038720 9545 consensus_queue.cc:1035] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:15.041728 8707 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:15.041837 8707 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:15.042483 8707 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5: Sending DeleteTablet for 3 replicas of tablet 2b3860a57e014645b6219568f1d430dd
I20260806 06:36:15.042742 8904 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3860a57e014645b6219568f1d430dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:15 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:53658
I20260806 06:36:15.042807 9312 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3860a57e014645b6219568f1d430dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:15 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:34386
I20260806 06:36:15.042865 9493 tablet_replica.cc:330] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298: stopping tablet replica
I20260806 06:36:15.042858 9120 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b3860a57e014645b6219568f1d430dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:15 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:45408
I20260806 06:36:15.042908 9491 tablet_replica.cc:330] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838: stopping tablet replica
I20260806 06:36:15.042930 9493 raft_consensus.cc:2229] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:15.042944 9492 tablet_replica.cc:330] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588: stopping tablet replica
I20260806 06:36:15.042953 9491 raft_consensus.cc:2229] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:36:15.042980 9493 raft_consensus.cc:2258] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:15.042999 9492 raft_consensus.cc:2229] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:15.043018 9491 raft_consensus.cc:2258] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:15.043061 9492 raft_consensus.cc:2258] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:15.043399 9493 ts_tablet_manager.cc:1905] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:15.043387 8707 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 06:36:15.043483 9491 ts_tablet_manager.cc:1905] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:15.043499 9492 ts_tablet_manager.cc:1905] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 06:36:15.043526 8707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:36:15.044680 9120 tablet_service.cc:1467] Processing CreateTablet for tablet a6b67a6dc8ef49ae8b044f61ce360ccc (DEFAULT_TABLE table=test-table [id=cc39f0dcea1545ce8cc051528d85f88a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:15.044785 9120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6b67a6dc8ef49ae8b044f61ce360ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:15.045177 9491 ts_tablet_manager.cc:1918] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:15.045233 9492 ts_tablet_manager.cc:1918] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:15.045234 9491 log.cc:1196] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/2b3860a57e014645b6219568f1d430dd
I20260806 06:36:15.045307 9492 log.cc:1196] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/2b3860a57e014645b6219568f1d430dd
I20260806 06:36:15.045356 9493 ts_tablet_manager.cc:1918] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:15.045428 9493 log.cc:1196] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-0/wal/wals/2b3860a57e014645b6219568f1d430dd
I20260806 06:36:15.045511 9491 ts_tablet_manager.cc:1939] T 2b3860a57e014645b6219568f1d430dd P 7c9e048616d347d7aafc8191fc2e9838: Deleting consensus metadata
I20260806 06:36:15.045547 9492 ts_tablet_manager.cc:1939] T 2b3860a57e014645b6219568f1d430dd P 98116dbe143a48979daa6ce5b8499588: Deleting consensus metadata
I20260806 06:36:15.045673 9493 ts_tablet_manager.cc:1939] T 2b3860a57e014645b6219568f1d430dd P 87816cb844af4ed5a225b36dfa101298: Deleting consensus metadata
I20260806 06:36:15.045221 9312 tablet_service.cc:1467] Processing CreateTablet for tablet a6b67a6dc8ef49ae8b044f61ce360ccc (DEFAULT_TABLE table=test-table [id=cc39f0dcea1545ce8cc051528d85f88a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:15.046031 9312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6b67a6dc8ef49ae8b044f61ce360ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:15.046111 9457 tablet_bootstrap.cc:492] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588: Bootstrap starting.
I20260806 06:36:15.046468 9457 tablet_bootstrap.cc:654] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:15.046821 8688 catalog_manager.cc:4841] TS 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911): tablet 2b3860a57e014645b6219568f1d430dd (table test-table [id=b0414e1b91b946b59b4ac70b819a203f]) successfully deleted
I20260806 06:36:15.046919 8688 catalog_manager.cc:4841] TS 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751): tablet 2b3860a57e014645b6219568f1d430dd (table test-table [id=b0414e1b91b946b59b4ac70b819a203f]) successfully deleted
I20260806 06:36:15.046995 8904 tablet_service.cc:1467] Processing CreateTablet for tablet a6b67a6dc8ef49ae8b044f61ce360ccc (DEFAULT_TABLE table=test-table [id=cc39f0dcea1545ce8cc051528d85f88a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:15.047108 8904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6b67a6dc8ef49ae8b044f61ce360ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:15.047168 9457 tablet_bootstrap.cc:492] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588: No bootstrap required, opened a new log
I20260806 06:36:15.047235 9457 ts_tablet_manager.cc:1397] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:15.047346 9457 raft_consensus.cc:348] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:15.047430 9457 raft_consensus.cc:374] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:15.047453 9457 raft_consensus.cc:729] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98116dbe143a48979daa6ce5b8499588, State: Initialized, Role: FOLLOWER
I20260806 06:36:15.047452 9456 tablet_bootstrap.cc:492] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838: Bootstrap starting.
I20260806 06:36:15.047501 9457 consensus_queue.cc:260] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:15.047602 9457 ts_tablet_manager.cc:1428] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:36:15.047768 9456 tablet_bootstrap.cc:654] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:15.048116 8690 catalog_manager.cc:4841] TS 7c9e048616d347d7aafc8191fc2e9838 (127.29.142.131:41265): tablet 2b3860a57e014645b6219568f1d430dd (table test-table [id=b0414e1b91b946b59b4ac70b819a203f]) successfully deleted
I20260806 06:36:15.048434 9459 tablet_bootstrap.cc:492] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298: Bootstrap starting.
I20260806 06:36:15.048583 9456 tablet_bootstrap.cc:492] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838: No bootstrap required, opened a new log
I20260806 06:36:15.048626 9456 ts_tablet_manager.cc:1397] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:15.048710 9456 raft_consensus.cc:348] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:15.048763 9456 raft_consensus.cc:374] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:15.048777 9456 raft_consensus.cc:729] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9e048616d347d7aafc8191fc2e9838, State: Initialized, Role: FOLLOWER
I20260806 06:36:15.048805 9459 tablet_bootstrap.cc:654] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:15.048831 9456 consensus_queue.cc:260] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:15.048938 9456 ts_tablet_manager.cc:1428] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:15.049530 9459 tablet_bootstrap.cc:492] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298: No bootstrap required, opened a new log
I20260806 06:36:15.049588 9459 ts_tablet_manager.cc:1397] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:15.049690 9459 raft_consensus.cc:348] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:15.049784 9459 raft_consensus.cc:374] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:15.049816 9459 raft_consensus.cc:729] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87816cb844af4ed5a225b36dfa101298, State: Initialized, Role: FOLLOWER
I20260806 06:36:15.049873 9459 consensus_queue.cc:260] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:15.049988 9459 ts_tablet_manager.cc:1428] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:15.367079 9467 raft_consensus.cc:482] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:36:15.367208 9467 raft_consensus.cc:504] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:15.367421 9467 leader_election.cc:288] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751), 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911)
I20260806 06:36:15.367702 8936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6b67a6dc8ef49ae8b044f61ce360ccc" candidate_uuid: "7c9e048616d347d7aafc8191fc2e9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87816cb844af4ed5a225b36dfa101298" is_pre_election: true
I20260806 06:36:15.367751 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6b67a6dc8ef49ae8b044f61ce360ccc" candidate_uuid: "7c9e048616d347d7aafc8191fc2e9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98116dbe143a48979daa6ce5b8499588" is_pre_election: true
I20260806 06:36:15.367843 8936 raft_consensus.cc:2454] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9e048616d347d7aafc8191fc2e9838 in term 0.
I20260806 06:36:15.367882 9153 raft_consensus.cc:2454] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9e048616d347d7aafc8191fc2e9838 in term 0.
I20260806 06:36:15.368027 9264 leader_election.cc:302] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c9e048616d347d7aafc8191fc2e9838, 87816cb844af4ed5a225b36dfa101298; no voters:
I20260806 06:36:15.368150 9467 raft_consensus.cc:2781] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:36:15.368221 9467 raft_consensus.cc:482] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:36:15.368239 9467 raft_consensus.cc:3037] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:15.369057 9467 raft_consensus.cc:504] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:15.369215 9467 leader_election.cc:288] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [CANDIDATE]: Term 1 election: Requested vote from peers 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751), 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911)
I20260806 06:36:15.369436 8936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6b67a6dc8ef49ae8b044f61ce360ccc" candidate_uuid: "7c9e048616d347d7aafc8191fc2e9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87816cb844af4ed5a225b36dfa101298"
I20260806 06:36:15.369529 8936 raft_consensus.cc:3037] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:15.369465 9153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6b67a6dc8ef49ae8b044f61ce360ccc" candidate_uuid: "7c9e048616d347d7aafc8191fc2e9838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98116dbe143a48979daa6ce5b8499588"
I20260806 06:36:15.369580 9153 raft_consensus.cc:3037] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:15.370386 8936 raft_consensus.cc:2454] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9e048616d347d7aafc8191fc2e9838 in term 1.
I20260806 06:36:15.370386 9153 raft_consensus.cc:2454] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9e048616d347d7aafc8191fc2e9838 in term 1.
I20260806 06:36:15.370605 9264 leader_election.cc:302] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c9e048616d347d7aafc8191fc2e9838, 87816cb844af4ed5a225b36dfa101298; no voters:
I20260806 06:36:15.370734 9467 raft_consensus.cc:2781] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:15.370802 9467 raft_consensus.cc:686] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 1 LEADER]: Becoming Leader. State: Replica: 7c9e048616d347d7aafc8191fc2e9838, State: Running, Role: LEADER
I20260806 06:36:15.370872 9467 consensus_queue.cc:237] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } }
I20260806 06:36:15.371601 8709 catalog_manager.cc:5495] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c9e048616d347d7aafc8191fc2e9838 (127.29.142.131). New cstate: current_term: 1 leader_uuid: "7c9e048616d347d7aafc8191fc2e9838" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c9e048616d347d7aafc8191fc2e9838" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 41265 } health_report { overall_health: HEALTHY } } }
I20260806 06:36:15.409088 9153 raft_consensus.cc:1261] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Refusing update from remote peer 7c9e048616d347d7aafc8191fc2e9838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:15.409091 8936 raft_consensus.cc:1261] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Refusing update from remote peer 7c9e048616d347d7aafc8191fc2e9838: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:15.409334 9500 consensus_queue.cc:1035] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87816cb844af4ed5a225b36dfa101298" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:15.409448 9467 consensus_queue.cc:1035] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98116dbe143a48979daa6ce5b8499588" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:15.419368 8707 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:15.419482 8707 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:15.420082 8707 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f3935f5d44f4420b8b7e6ff954a82e5: Sending DeleteTablet for 3 replicas of tablet a6b67a6dc8ef49ae8b044f61ce360ccc
I20260806 06:36:15.420369 8904 tablet_service.cc:1514] Processing DeleteTablet for tablet a6b67a6dc8ef49ae8b044f61ce360ccc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:15 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:53658
I20260806 06:36:15.420369 9312 tablet_service.cc:1514] Processing DeleteTablet for tablet a6b67a6dc8ef49ae8b044f61ce360ccc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:15 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:34386
I20260806 06:36:15.420476 9493 tablet_replica.cc:330] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298: stopping tablet replica
I20260806 06:36:15.420531 9493 raft_consensus.cc:2229] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:15.420516 9491 tablet_replica.cc:330] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838: stopping tablet replica
I20260806 06:36:15.420580 9493 raft_consensus.cc:2258] T a6b67a6dc8ef49ae8b044f61ce360ccc P 87816cb844af4ed5a225b36dfa101298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:15.420553 9120 tablet_service.cc:1514] Processing DeleteTablet for tablet a6b67a6dc8ef49ae8b044f61ce360ccc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:15 UTC) from {username='oryx', principal='oryx/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:45408
I20260806 06:36:15.420601 9491 raft_consensus.cc:2229] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:36:15.420657 9492 tablet_replica.cc:330] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588: stopping tablet replica
I20260806 06:36:15.420673 9491 raft_consensus.cc:2258] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:15.420727 9492 raft_consensus.cc:2229] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:15.420764 9492 raft_consensus.cc:2258] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:15.420992 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 8815
I20260806 06:36:15.421133 9492 ts_tablet_manager.cc:1905] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:15.421167 9491 ts_tablet_manager.cc:1905] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:15.422920 9492 ts_tablet_manager.cc:1918] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:36:15.423000 9492 log.cc:1196] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/a6b67a6dc8ef49ae8b044f61ce360ccc
I20260806 06:36:15.423056 9491 ts_tablet_manager.cc:1918] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:36:15.423126 9491 log.cc:1196] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/a6b67a6dc8ef49ae8b044f61ce360ccc
I20260806 06:36:15.423255 9492 ts_tablet_manager.cc:1939] T a6b67a6dc8ef49ae8b044f61ce360ccc P 98116dbe143a48979daa6ce5b8499588: Deleting consensus metadata
I20260806 06:36:15.423394 9491 ts_tablet_manager.cc:1939] T a6b67a6dc8ef49ae8b044f61ce360ccc P 7c9e048616d347d7aafc8191fc2e9838: Deleting consensus metadata
I20260806 06:36:15.423638 8688 catalog_manager.cc:4841] TS 98116dbe143a48979daa6ce5b8499588 (127.29.142.130:42911): tablet a6b67a6dc8ef49ae8b044f61ce360ccc (table test-table [id=cc39f0dcea1545ce8cc051528d85f88a]) successfully deleted
I20260806 06:36:15.423719 8690 catalog_manager.cc:4841] TS 7c9e048616d347d7aafc8191fc2e9838 (127.29.142.131:41265): tablet a6b67a6dc8ef49ae8b044f61ce360ccc (table test-table [id=cc39f0dcea1545ce8cc051528d85f88a]) successfully deleted
W20260806 06:36:15.434741 8688 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:46751 (error 108)
I20260806 06:36:15.435051 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 9016
I20260806 06:36:15.445367 9600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:15.435303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:46751 (local address 127.0.0.1:53676)
0806 06:36:15.445165 (+ 9862us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:15.445297 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:46751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9831,"thread_start_us":39,"threads_started":1}
W20260806 06:36:15.445526 8688 catalog_manager.cc:4584] TS 87816cb844af4ed5a225b36dfa101298 (127.29.142.129:46751): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6b67a6dc8ef49ae8b044f61ce360ccc: Network error: Client connection negotiation failed: client connection to 127.29.142.129:46751: connect: Connection refused (error 111)
I20260806 06:36:15.451853 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 9234
I20260806 06:36:15.468461 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 8663
2026-08-06T06:36:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3334 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:36:15 dist-test-slave-3kk6 krb5kdc[9607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:36:15 dist-test-slave-3kk6 krb5kdc[9607](info): set up 2 sockets
Aug 06 06:36:15 dist-test-slave-3kk6 krb5kdc[9607](info): commencing operation
krb5kdc: starting...
W20260806 06:36:16.725443 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:36:16 dist-test-slave-3kk6 krb5kdc[9607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:36:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:36:16.853000 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:36811
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:36811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:16.941087 9624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:16.941390 9624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:16.941438 9624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:16.942888 9624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:16.942966 9624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:16.942993 9624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:16.943018 9624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:16.943050 9624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:16.944832 9624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:36811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:36811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:16.945160 9624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:16.945426 9624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:36:16.951016 9624 server_base.cc:1034] running on GCE node
W20260806 06:36:16.950925 9633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:16.950933 9631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:16.950925 9630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:16.951505 9624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:16.951865 9624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:16.953059 9624 hybrid_clock.cc:648] HybridClock initialized: now 1785998176953040 us; error 49 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T06:36:16Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1483 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:36:17 dist-test-slave-3kk6 krb5kdc[9641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:36:17 dist-test-slave-3kk6 krb5kdc[9641](info): set up 2 sockets
Aug 06 06:36:17 dist-test-slave-3kk6 krb5kdc[9641](info): commencing operation
krb5kdc: starting...
W20260806 06:36:18.192442 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:36:18 dist-test-slave-3kk6 krb5kdc[9641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:36:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:36:18.318243 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:33211
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:33211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:18.403247 9660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:18.403528 9660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:18.403571 9660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:18.404806 9660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:18.404886 9660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:18.404908 9660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:18.404929 9660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:18.404956 9660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:18.406507 9660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:33211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:33211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:18.406845 9660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:18.407099 9660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:18.412207 9674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:18.412205 9669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:18.412205 9671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:18.412524 9660 server_base.cc:1034] running on GCE node
I20260806 06:36:18.412777 9660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:18.413049 9660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:18.414192 9660 hybrid_clock.cc:648] HybridClock initialized: now 1785998178414182 us; error 32 us; skew 500 ppm
Aug 06 06:36:18 dist-test-slave-3kk6 krb5kdc[9641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998178, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:18.416937 9660 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:36:18.417356 9660 webserver.cc:460] Webserver started at http://127.29.142.190:34359/ using document root <none> and password file <none>
I20260806 06:36:18.417553 9660 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:18.417598 9660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:18.417695 9660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:18.418838 9660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "0cc1af6ff0274808a05e2b7bed661fb3"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "e50e769cadb3fefccb071be931eef15d"
server_key_iv: "49476ae021017767ba077d1d4728aff0"
server_key_version: "encryptionkey@0"
I20260806 06:36:18.419183 9660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "0cc1af6ff0274808a05e2b7bed661fb3"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "e50e769cadb3fefccb071be931eef15d"
server_key_iv: "49476ae021017767ba077d1d4728aff0"
server_key_version: "encryptionkey@0"
I20260806 06:36:18.420539 9660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:36:18.421303 9683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:18.421501 9660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:18.421599 9660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "0cc1af6ff0274808a05e2b7bed661fb3"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "e50e769cadb3fefccb071be931eef15d"
server_key_iv: "49476ae021017767ba077d1d4728aff0"
server_key_version: "encryptionkey@0"
I20260806 06:36:18.421684 9660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:18.431733 9660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:18.434123 9660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:18.434304 9660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:18.438575 9660 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:33211
I20260806 06:36:18.438607 9756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:33211 every 8 connection(s)
I20260806 06:36:18.439186 9660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:18.439888 9757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:18.442287 9757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3: Bootstrap starting.
I20260806 06:36:18.442976 9757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:18.443276 9757 log.cc:824] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:18.443876 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 9660
I20260806 06:36:18.444000 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:18.444141 9757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3: No bootstrap required, opened a new log
I20260806 06:36:18.444204 30266 external_mini_cluster.cc:1428] Setting key cf245cb68799d4d6e12d31c31bc4db77
I20260806 06:36:18.446604 9757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33211 } }
I20260806 06:36:18.446815 9757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:18.446846 9757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cc1af6ff0274808a05e2b7bed661fb3, State: Initialized, Role: FOLLOWER
I20260806 06:36:18.446991 9757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33211 } }
I20260806 06:36:18.447065 9757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:18.447115 9757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:18.447165 9757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:18.447757 9757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33211 } }
I20260806 06:36:18.447857 9757 leader_election.cc:302] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cc1af6ff0274808a05e2b7bed661fb3; no voters:
I20260806 06:36:18.448060 9757 leader_election.cc:288] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:18.448094 9763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:18.448215 9763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 1 LEADER]: Becoming Leader. State: Replica: 0cc1af6ff0274808a05e2b7bed661fb3, State: Running, Role: LEADER
I20260806 06:36:18.448309 9763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33211 } }
I20260806 06:36:18.448344 9757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:18.448647 9763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33211 } } }
I20260806 06:36:18.448809 9763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:18.449013 9763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cc1af6ff0274808a05e2b7bed661fb3. Latest consensus state: current_term: 1 leader_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33211 } } }
I20260806 06:36:18.449123 9763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:18.449497 9779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:18.450054 9779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:18.452081 9779 catalog_manager.cc:1347] Generated new cluster ID: 71ac05c2ab3d4cc09e95e8177827bc43
I20260806 06:36:18.452135 9779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:36:18 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:18.457010 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:18.444885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43442 (local address 127.29.142.190:33211)
0806 06:36:18.445059 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:18.445068 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:18.445093 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:36:18.445521 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:18.445527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:18.445587 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:18.445718 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:18.446567 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.448096 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.449288 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.449921 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.450040 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:18.452953 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:18.452958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:18.452958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:18.452971 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:18.455161 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:18.455556 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:18.455560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:18.455560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:18.455675 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:18.455886 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:18.455887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:18.455887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:18.456229 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:18.456399 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:18.456482 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 06:36:18.456630 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260806 06:36:18.475117 9779 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:18.475899 9779 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:18.485483 9779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3: Generated new TSK 0
I20260806 06:36:18.485759 9779 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:36:18.517210 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:33211
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:18.603109 9803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:18.603336 9803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:18.603360 9803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:18.604482 9803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:18.604521 9803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:18.604547 9803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:36:18.605954 9803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:33211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:18.606159 9803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:18.606339 9803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:18.611464 9815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:18.611474 9811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:18.611476 9812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:18.611868 9803 server_base.cc:1034] running on GCE node
I20260806 06:36:18.612103 9803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:18.612310 9803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:18.613448 9803 hybrid_clock.cc:648] HybridClock initialized: now 1785998178613435 us; error 32 us; skew 500 ppm
Aug 06 06:36:18 dist-test-slave-3kk6 krb5kdc[9641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998178, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:18.616098 9803 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:36:18.616503 9803 webserver.cc:460] Webserver started at http://127.29.142.129:41759/ using document root <none> and password file <none>
I20260806 06:36:18.616717 9803 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:18.616760 9803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:18.616852 9803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:18.617877 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "98c62eae9e5a46b39a3dfe9b175b6041"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "32fcdb8a10eed2dd45062f5e003ffefa"
server_key_iv: "acb67403a109b1f0764b5dca88adaf7c"
server_key_version: "encryptionkey@0"
I20260806 06:36:18.618186 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "98c62eae9e5a46b39a3dfe9b175b6041"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "32fcdb8a10eed2dd45062f5e003ffefa"
server_key_iv: "acb67403a109b1f0764b5dca88adaf7c"
server_key_version: "encryptionkey@0"
I20260806 06:36:18.619534 9803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:36:18.620234 9827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:18.620407 9803 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:36:18.620512 9803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "98c62eae9e5a46b39a3dfe9b175b6041"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "32fcdb8a10eed2dd45062f5e003ffefa"
server_key_iv: "acb67403a109b1f0764b5dca88adaf7c"
server_key_version: "encryptionkey@0"
I20260806 06:36:18.620585 9803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:18.637458 9803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:18.639552 9803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:18.639755 9803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:18.640020 9803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:18.640344 9803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:18.640398 9803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:18.640440 9803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:18.640473 9803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:18.646169 9803 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:33597
I20260806 06:36:18.646245 9985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:33597 every 8 connection(s)
I20260806 06:36:18.646586 9803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:36:18 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998178, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:18.652987 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 9803
I20260806 06:36:18.653117 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:36:18.653337 30266 external_mini_cluster.cc:1428] Setting key 18d6f1a03ac4f8f76f2c05742a15d4d0
I20260806 06:36:18.656417 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:18.647377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:42421 (local address 127.29.142.190:33211)
0806 06:36:18.647457 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:18.647460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:18.647635 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:36:18.648130 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:18.648131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:18.648142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:18.648251 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:18.649377 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.649902 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.650950 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.651148 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.651213 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:18.653513 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:18.653515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:18.653515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:18.653524 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:18.655015 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:18.655444 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:18.655445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:18.655446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:18.655505 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:18.655767 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:18.655768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:18.655768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:18.655880 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:18.655949 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:18.656231 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 06:36:18.656323 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:36:18.656636 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:18.647332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:42421)
0806 06:36:18.647509 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:18.647547 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:18.648045 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:18.648343 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:18.648346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:18.648969 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:18.649189 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:18.649194 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.649984 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.649984 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:18.650873 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:18.650876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.651227 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.651228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:18.651333 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:18.651915 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:18.651924 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:18.653412 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:18.655113 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:18.655121 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:18.655123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:18.655354 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:18.655577 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:18.655578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:18.655578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:18.655695 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:18.655953 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:18.655960 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:18.656099 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 06:36:18.656210 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 06:36:18.656374 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260806 06:36:18.657151 9986 heartbeater.cc:347] Connected to a master server at 127.29.142.190:33211
I20260806 06:36:18.657261 9986 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:18.657462 9986 heartbeater.cc:511] Master 127.29.142.190:33211 requested a full tablet report, sending...
I20260806 06:36:18.657984 9704 ts_manager.cc:194] Registered new tserver with Master: 98c62eae9e5a46b39a3dfe9b175b6041 (127.29.142.129:33597)
I20260806 06:36:18.658645 9704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:42421
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:36:18.679754 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:33211
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:18.764539 9993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:18.764780 9993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:18.764806 9993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:18.766530 9993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:18.766587 9993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:18.766614 9993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:36:18.768160 9993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:33211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:18.768441 9993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:18.768695 9993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:18.773965 10003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:18.774034 10001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:18.773993 10000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:18.774257 9993 server_base.cc:1034] running on GCE node
I20260806 06:36:18.774523 9993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:18.774833 9993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:18.776013 9993 hybrid_clock.cc:648] HybridClock initialized: now 1785998178775988 us; error 40 us; skew 500 ppm
Aug 06 06:36:18 dist-test-slave-3kk6 krb5kdc[9641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998178, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:18.778851 9993 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:36:18.779347 9993 webserver.cc:460] Webserver started at http://127.29.142.130:36259/ using document root <none> and password file <none>
I20260806 06:36:18.779610 9993 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:18.779680 9993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:18.779836 9993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:18.781001 9993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "ca4da0841d6040c6a5bf6941654310e2"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "2f93eefe557b765acac10d78535d5df9"
server_key_iv: "49dc028afde4404eb51536ab7fa6c32a"
server_key_version: "encryptionkey@0"
I20260806 06:36:18.781385 9993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "ca4da0841d6040c6a5bf6941654310e2"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "2f93eefe557b765acac10d78535d5df9"
server_key_iv: "49dc028afde4404eb51536ab7fa6c32a"
server_key_version: "encryptionkey@0"
I20260806 06:36:18.782856 9993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:18.783603 10014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:18.783772 9993 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:18.783874 9993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "ca4da0841d6040c6a5bf6941654310e2"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "2f93eefe557b765acac10d78535d5df9"
server_key_iv: "49dc028afde4404eb51536ab7fa6c32a"
server_key_version: "encryptionkey@0"
I20260806 06:36:18.783953 9993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:18.791930 9993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:18.794513 9993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:18.794739 9993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:18.794996 9993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:18.795353 9993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:18.795414 9993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:18.795461 9993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:18.795499 9993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:18.801734 9993 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:43027
I20260806 06:36:18.801890 10177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:43027 every 8 connection(s)
I20260806 06:36:18.802217 9993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:36:18.805631 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 9993
I20260806 06:36:18.805783 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:36:18.805996 30266 external_mini_cluster.cc:1428] Setting key 05b9c4d47f515c70e0eb2752797777d3
Aug 06 06:36:18 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998178, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:18.813162 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:18.803173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:49793 (local address 127.29.142.190:33211)
0806 06:36:18.803257 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:18.803259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:18.803426 (+ 167us) server_negotiation.cc:407] Connection header received
0806 06:36:18.803982 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:18.803984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:18.803996 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:18.804058 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:18.805102 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.805633 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.807158 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.807346 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.807414 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:18.810051 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:18.810052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:18.810052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:18.810059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:18.811610 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:18.812134 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:18.812135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:18.812135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:18.812188 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:18.812494 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:18.812495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:18.812495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:18.812601 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:18.812653 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:18.813026 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 06:36:18.813093 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:36:18.813585 10183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:18.803114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:49793)
0806 06:36:18.803286 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:18.803327 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:18.803862 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:18.804120 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:18.804123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:18.804779 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:18.804995 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:18.805001 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.805763 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.805766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:18.807076 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:18.807080 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.807436 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.807437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:18.807595 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:18.808242 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:18.808253 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:18.809963 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:18.811706 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:18.811715 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:18.811718 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:18.812057 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:18.812259 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:18.812259 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:18.812260 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:18.812423 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:18.812668 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:18.812676 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:18.812868 (+ 192us) client_negotiation.cc:769] Sending connection context
0806 06:36:18.812997 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 06:36:18.813192 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260806 06:36:18.814213 10179 heartbeater.cc:347] Connected to a master server at 127.29.142.190:33211
I20260806 06:36:18.814396 10179 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:18.814599 10179 heartbeater.cc:511] Master 127.29.142.190:33211 requested a full tablet report, sending...
I20260806 06:36:18.815088 9704 ts_manager.cc:194] Registered new tserver with Master: ca4da0841d6040c6a5bf6941654310e2 (127.29.142.130:43027)
I20260806 06:36:18.815498 9704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:49793
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:36:18.837858 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:33211
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:18.927457 10186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:18.927729 10186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:18.927758 10186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:18.929026 10186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:18.929070 10186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:18.929100 10186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:36:18.931380 10186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:33211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:18.931666 10186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:18.931962 10186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:18.937532 10199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:18.937541 10193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:18.937532 10195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:18.937696 10186 server_base.cc:1034] running on GCE node
I20260806 06:36:18.937990 10186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:18.938262 10186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:18.939438 10186 hybrid_clock.cc:648] HybridClock initialized: now 1785998178939432 us; error 45 us; skew 500 ppm
Aug 06 06:36:18 dist-test-slave-3kk6 krb5kdc[9641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998178, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:18.942219 10186 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:36:18.942795 10186 webserver.cc:460] Webserver started at http://127.29.142.131:40861/ using document root <none> and password file <none>
I20260806 06:36:18.943068 10186 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:18.943143 10186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:18.943300 10186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:18.944522 10186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "8536abe496a343798c66bb3ec3ac143d"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "5a52e9c456f8374223df707bb7f56d90"
server_key_iv: "90d58eb41acb20af2bdcd3db722a1770"
server_key_version: "encryptionkey@0"
I20260806 06:36:18.944932 10186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "8536abe496a343798c66bb3ec3ac143d"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "5a52e9c456f8374223df707bb7f56d90"
server_key_iv: "90d58eb41acb20af2bdcd3db722a1770"
server_key_version: "encryptionkey@0"
I20260806 06:36:18.946424 10186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:36:18.947422 10210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:18.947656 10186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:18.947774 10186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "8536abe496a343798c66bb3ec3ac143d"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "5a52e9c456f8374223df707bb7f56d90"
server_key_iv: "90d58eb41acb20af2bdcd3db722a1770"
server_key_version: "encryptionkey@0"
I20260806 06:36:18.947860 10186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:18.956960 10186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:18.959558 10186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:18.959746 10186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:18.959980 10186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:18.960304 10186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:18.960337 10186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:18.960379 10186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:18.960408 10186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:18.966385 10186 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:33397
I20260806 06:36:18.966478 10380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:33397 every 8 connection(s)
I20260806 06:36:18.966869 10186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:36:18 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998178, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:18.974160 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 10186
I20260806 06:36:18.974260 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:36:18.974444 30266 external_mini_cluster.cc:1428] Setting key 7078c3ee7cd21d6809f55a519ddf47ba
I20260806 06:36:18.976527 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:18.967629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:46859 (local address 127.29.142.190:33211)
0806 06:36:18.967682 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:18.967684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:18.967910 (+ 226us) server_negotiation.cc:407] Connection header received
0806 06:36:18.968473 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:18.968475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:18.968489 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:18.968556 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:18.969557 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.970125 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.971196 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.971405 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.971496 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:18.973709 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:18.973711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:18.973711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:18.973719 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:18.975387 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:18.975760 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:18.975761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:18.975761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:18.975812 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:18.975996 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:18.975997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:18.975997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:18.976100 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:18.976153 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:18.976399 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 06:36:18.976461 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:36:18.976697 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:18.967596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:46859)
0806 06:36:18.967757 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:18.967804 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:18.968354 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:18.968630 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:18.968634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:18.969243 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:18.969442 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:18.969447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.970238 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.970239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:18.971089 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:18.971091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:18.971509 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:18.971511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:18.971666 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:18.972176 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:18.972185 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:18.973598 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:18.975460 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:18.975465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:18.975467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:18.975690 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:18.975854 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:18.975855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:18.975855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:18.975947 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:18.976161 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:18.976166 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:18.976285 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 06:36:18.976371 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:36:18.976496 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260806 06:36:18.977190 10381 heartbeater.cc:347] Connected to a master server at 127.29.142.190:33211
I20260806 06:36:18.977298 10381 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:18.977504 10381 heartbeater.cc:511] Master 127.29.142.190:33211 requested a full tablet report, sending...
I20260806 06:36:18.977874 9704 ts_manager.cc:194] Registered new tserver with Master: 8536abe496a343798c66bb3ec3ac143d (127.29.142.131:33397)
I20260806 06:36:18.978289 9704 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:46859
I20260806 06:36:18.986195 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab.
Entry for principal oryx/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab.
WARNING: no policy specified for oryx/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab.
Entry for principal oryx/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab.
WARNING: no policy specified for oryx/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab.
Entry for principal oryx/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab.
I20260806 06:36:19.066637 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 9803
I20260806 06:36:19.091631 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 9993
I20260806 06:36:19.101122 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 10186
I20260806 06:36:19.110750 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 9660
I20260806 06:36:19.120355 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:33211
--webserver_interface=127.29.142.190
--webserver_port=34359
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:33211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:19.205114 10391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:19.205359 10391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:19.205384 10391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:19.206571 10391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:19.206622 10391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:19.206636 10391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:19.206702 10391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:19.206723 10391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:19.208266 10391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:33211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:33211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=34359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:19.208549 10391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:19.208803 10391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:19.214151 10398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:19.214144 10402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:19.214151 10400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:19.214478 10391 server_base.cc:1034] running on GCE node
I20260806 06:36:19.214803 10391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:19.215147 10391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:19.216343 10391 hybrid_clock.cc:648] HybridClock initialized: now 1785998179216336 us; error 67 us; skew 500 ppm
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998179, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:19.219151 10391 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:36:19.219678 10391 webserver.cc:460] Webserver started at http://127.29.142.190:34359/ using document root <none> and password file <none>
I20260806 06:36:19.219882 10391 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:19.219940 10391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:19.221374 10391 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:19.222090 10412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:19.222295 10391 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:19.222409 10391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "0cc1af6ff0274808a05e2b7bed661fb3"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "e50e769cadb3fefccb071be931eef15d"
server_key_iv: "49476ae021017767ba077d1d4728aff0"
server_key_version: "encryptionkey@0"
I20260806 06:36:19.222862 10391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:19.237677 10391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:19.240361 10391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:19.240509 10391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:19.244935 10391 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:33211
I20260806 06:36:19.244973 10481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:33211 every 8 connection(s)
I20260806 06:36:19.245409 10391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:19.245986 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 10391
I20260806 06:36:19.246340 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:33597
--local_ip_for_outbound_sockets=127.29.142.129
--tserver_master_addrs=127.29.142.190:33211
--webserver_port=41759
--webserver_interface=127.29.142.129
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.142.129
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:36:19.247865 10483 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:36:19.248564 10483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3: Bootstrap starting.
I20260806 06:36:19.259919 10483 log.cc:824] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:19.262207 10483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:36:19.262585 10483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3: Bootstrap complete.
I20260806 06:36:19.265905 10483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33211 } }
I20260806 06:36:19.266253 10483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cc1af6ff0274808a05e2b7bed661fb3, State: Initialized, Role: FOLLOWER
I20260806 06:36:19.266441 10483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33211 } }
I20260806 06:36:19.266515 10483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:19.266566 10483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:19.266614 10483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:36:19.268157 10483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33211 } }
I20260806 06:36:19.268290 10483 leader_election.cc:302] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cc1af6ff0274808a05e2b7bed661fb3; no voters:
I20260806 06:36:19.268491 10483 leader_election.cc:288] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:36:19.268585 10491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:36:19.268747 10483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:19.268771 10491 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [term 2 LEADER]: Becoming Leader. State: Replica: 0cc1af6ff0274808a05e2b7bed661fb3, State: Running, Role: LEADER
I20260806 06:36:19.268864 10491 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33211 } }
I20260806 06:36:19.269160 10492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33211 } } }
I20260806 06:36:19.269250 10492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:19.269521 10493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cc1af6ff0274808a05e2b7bed661fb3. Latest consensus state: current_term: 2 leader_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc1af6ff0274808a05e2b7bed661fb3" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33211 } } }
I20260806 06:36:19.269665 10493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:19.269840 10502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:19.270396 10502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:19.270543 10502 catalog_manager.cc:1259] Loaded cluster ID: 71ac05c2ab3d4cc09e95e8177827bc43
I20260806 06:36:19.270828 10502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:36:19.272039 10502 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:19.272224 10502 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0cc1af6ff0274808a05e2b7bed661fb3: Loaded TSK: 0
I20260806 06:36:19.272442 10502 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:36:19.341450 10487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:19.341746 10487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:19.341795 10487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:19.343139 10487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:19.343216 10487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:19.343266 10487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:36:19.344864 10487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.129.keytab
--principal=oryx/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:33597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=41759
--tserver_master_addrs=127.29.142.190:33211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:19.345172 10487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:19.345438 10487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:19.351902 10524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:19.351922 10522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:19.352021 10521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:19.352106 10487 server_base.cc:1034] running on GCE node
I20260806 06:36:19.352455 10487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:19.352770 10487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:19.353971 10487 hybrid_clock.cc:648] HybridClock initialized: now 1785998179353954 us; error 31 us; skew 500 ppm
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998179, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:19.357331 10487 init.cc:377] Logged in from keytab as oryx/127.29.142.129@KRBTEST.COM (short username oryx)
I20260806 06:36:19.357923 10487 webserver.cc:460] Webserver started at http://127.29.142.129:41759/ using document root <none> and password file <none>
I20260806 06:36:19.358179 10487 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:19.358283 10487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:19.359886 10487 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:19.360607 10535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:19.360810 10487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:19.360905 10487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "98c62eae9e5a46b39a3dfe9b175b6041"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "32fcdb8a10eed2dd45062f5e003ffefa"
server_key_iv: "acb67403a109b1f0764b5dca88adaf7c"
server_key_version: "encryptionkey@0"
I20260806 06:36:19.361263 10487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:19.370750 10487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:19.373294 10487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:19.373473 10487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:19.373742 10487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:19.374059 10487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:19.374114 10487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:19.374159 10487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:19.374184 10487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:19.380725 10487 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:33597
I20260806 06:36:19.380764 10690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:33597 every 8 connection(s)
I20260806 06:36:19.381163 10487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:36:19.383158 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 10487
I20260806 06:36:19.383512 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:43027
--local_ip_for_outbound_sockets=127.29.142.130
--tserver_master_addrs=127.29.142.190:33211
--webserver_port=36259
--webserver_interface=127.29.142.130
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.142.130
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:19.391103 10693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:19.382073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:57745)
0806 06:36:19.382261 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:19.382305 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:19.382944 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:19.383422 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:19.383426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:19.384221 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:19.384422 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.384428 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.385740 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.385742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.386596 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.386599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.387269 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.387270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.387411 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.388071 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:19.388081 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:19.390846 (+ 2765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
W20260806 06:36:19.391213 10696 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:19.382174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:57745 (local address 127.29.142.190:33211)
0806 06:36:19.382392 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:19.382400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:19.382443 (+ 43us) server_negotiation.cc:407] Connection header received
0806 06:36:19.383169 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:19.383174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:19.383240 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:19.383379 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:19.384553 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.385615 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.386737 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.387167 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.387295 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.391013 (+ 3718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:57745: BlockingRecv error: recv got EOF from 127.29.142.129:57745 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":122,"threads_started":1}
W20260806 06:36:19.391719 10691 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:33211 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.142.190:33211: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:19.396623 10693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:19.392045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:59493)
0806 06:36:19.392095 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:19.392106 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:19.392151 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:19.392292 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:19.392292 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:19.392443 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:19.392493 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.392494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.393165 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.393166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.393690 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.393692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.394093 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.394093 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.394161 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.394697 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:19.394702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:19.396529 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 06:36:19.396682 10696 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:19.392085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:59493 (local address 127.29.142.190:33211)
0806 06:36:19.392123 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:19.392125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:19.392136 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:19.392208 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:19.392209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:19.392220 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:19.392287 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:19.392577 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.393075 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.393781 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.393998 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.394083 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.396613 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:59493: BlockingRecv error: recv got EOF from 127.29.142.129:59493 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:19.401543 10693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:19.397163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:52787)
0806 06:36:19.397257 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:19.397269 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:19.397318 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:19.397446 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:19.397447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:19.397578 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:19.397608 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.397609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.398232 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.398233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.398882 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.398884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.399246 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.399247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.399300 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.399760 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:19.399766 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:19.401452 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":78}
W20260806 06:36:19.401588 10696 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:19.397217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:52787 (local address 127.29.142.190:33211)
0806 06:36:19.397256 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:19.397259 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:19.397301 (+ 42us) server_negotiation.cc:407] Connection header received
0806 06:36:19.397374 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:19.397375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:19.397386 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:19.397432 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:19.397685 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.398145 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.398969 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.399159 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.399245 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.401524 (+ 2279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:52787: BlockingRecv error: recv got EOF from 127.29.142.129:52787 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:36:19.402038 10691 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:36:19.482352 10697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:19.482592 10697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:19.482618 10697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:19.483855 10697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:19.483907 10697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:19.483937 10697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:36:19.485363 10697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.130.keytab
--principal=oryx/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:43027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=36259
--tserver_master_addrs=127.29.142.190:33211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:19.485563 10697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:19.485759 10697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:19.491158 10707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:19.491153 10711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:19.491151 10708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:19.491770 10697 server_base.cc:1034] running on GCE node
I20260806 06:36:19.492035 10697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:19.492308 10697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:19.493456 10697 hybrid_clock.cc:648] HybridClock initialized: now 1785998179493438 us; error 41 us; skew 500 ppm
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998179, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:19.496165 10697 init.cc:377] Logged in from keytab as oryx/127.29.142.130@KRBTEST.COM (short username oryx)
I20260806 06:36:19.496644 10697 webserver.cc:460] Webserver started at http://127.29.142.130:36259/ using document root <none> and password file <none>
I20260806 06:36:19.496886 10697 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:19.496974 10697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:19.498360 10697 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:19.499038 10720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:19.499212 10697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:19.499331 10697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "ca4da0841d6040c6a5bf6941654310e2"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "2f93eefe557b765acac10d78535d5df9"
server_key_iv: "49dc028afde4404eb51536ab7fa6c32a"
server_key_version: "encryptionkey@0"
I20260806 06:36:19.499702 10697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:19.515527 10697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:19.517982 10697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:19.518175 10697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:19.518461 10697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:19.518867 10697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:19.518927 10697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:19.518965 10697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:19.518992 10697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:19.525358 10697 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:43027
I20260806 06:36:19.525401 10880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:43027 every 8 connection(s)
I20260806 06:36:19.525801 10697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:36:19.530815 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 10697
I20260806 06:36:19.531288 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:33397
--local_ip_for_outbound_sockets=127.29.142.131
--tserver_master_addrs=127.29.142.190:33211
--webserver_port=40861
--webserver_interface=127.29.142.131
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.142.131
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:19.533834 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:19.526653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:33561)
0806 06:36:19.526844 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:19.526883 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:19.527384 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:19.527609 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:19.527613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:19.528215 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:19.528429 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.528434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.529115 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.529115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.529925 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.529927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.530297 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.530298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.530403 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.531129 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:19.531139 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:19.533660 (+ 2521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
W20260806 06:36:19.533895 10696 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:19.526684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:33561 (local address 127.29.142.190:33211)
0806 06:36:19.526742 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:19.526745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:19.526959 (+ 214us) server_negotiation.cc:407] Connection header received
0806 06:36:19.527481 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:19.527482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:19.527497 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:19.527554 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:19.528523 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.529027 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.530013 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.530208 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.530267 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.533822 (+ 3555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:33561: BlockingRecv error: recv got EOF from 127.29.142.130:33561 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 06:36:19.534159 10882 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:33211 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.142.190:33211: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:19.539264 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:19.534484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:46699)
0806 06:36:19.534528 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:19.534541 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:19.534610 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:19.534780 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:19.534781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:19.534924 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:19.534975 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.534977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.535648 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.535649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.536162 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.536163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.536548 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.536549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.536620 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.537185 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:19.537191 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:19.539159 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 06:36:19.539358 10696 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:19.534542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:46699 (local address 127.29.142.190:33211)
0806 06:36:19.534609 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:19.534612 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:19.534622 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:36:19.534701 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:19.534703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:19.534715 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:19.534773 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:19.535060 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.535561 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.536251 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.536457 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.536547 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.539277 (+ 2730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:46699: BlockingRecv error: recv got EOF from 127.29.142.130:46699 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:19.545185 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:19.539936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:52175)
0806 06:36:19.540073 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:19.540087 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:19.540158 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:19.540322 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:19.540324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:19.540502 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:19.540557 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.540560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.541210 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.541211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.542120 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.542124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.542502 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.542503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.542571 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.543079 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:19.543085 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:19.545086 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113}
W20260806 06:36:19.545266 10696 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:19.540017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:52175 (local address 127.29.142.190:33211)
0806 06:36:19.540076 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:19.540078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:19.540147 (+ 69us) server_negotiation.cc:407] Connection header received
0806 06:36:19.540226 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:19.540227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:19.540236 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:19.540284 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:19.540645 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.541118 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.542233 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.542417 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.542477 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.545182 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:52175: BlockingRecv error: recv got EOF from 127.29.142.130:52175 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 06:36:19.545547 10882 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:36:19.620353 10886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:19.620626 10886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:19.620678 10886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:19.621932 10886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:19.622013 10886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:19.622062 10886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:36:19.623670 10886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/krb5kdc/oryx_127.29.142.131.keytab
--principal=oryx/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:33397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=40861
--tserver_master_addrs=127.29.142.190:33211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:19.623973 10886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:19.624228 10886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:36:19.629987 10886 server_base.cc:1034] running on GCE node
W20260806 06:36:19.629980 10895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:19.629942 10894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:19.629954 10898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:19.630367 10886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:19.630635 10886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:19.631863 10886 hybrid_clock.cc:648] HybridClock initialized: now 1785998179631865 us; error 56 us; skew 500 ppm
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998179, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:19.634567 10886 init.cc:377] Logged in from keytab as oryx/127.29.142.131@KRBTEST.COM (short username oryx)
I20260806 06:36:19.635121 10886 webserver.cc:460] Webserver started at http://127.29.142.131:40861/ using document root <none> and password file <none>
I20260806 06:36:19.635380 10886 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:19.635470 10886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:19.636900 10886 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:19.637532 10908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:19.637708 10886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:19.637820 10886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "8536abe496a343798c66bb3ec3ac143d"
format_stamp: "Formatted at 2026-08-06 06:36:18 on dist-test-slave-3kk6"
server_key: "5a52e9c456f8374223df707bb7f56d90"
server_key_iv: "90d58eb41acb20af2bdcd3db722a1770"
server_key_version: "encryptionkey@0"
I20260806 06:36:19.638204 10886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:19.654167 10886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:19.656448 10886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:19.656633 10886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:19.656895 10886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:19.657239 10886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:19.657302 10886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:19.657346 10886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:19.657384 10886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:19.663884 10886 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:33397
I20260806 06:36:19.663932 11062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:33397 every 8 connection(s)
I20260806 06:36:19.664347 10886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:36:19.667834 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 10886
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:19.673099 11066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:19.665176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:47685)
0806 06:36:19.665373 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:19.665424 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:19.665930 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:19.666149 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:19.666153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:19.666796 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:19.667010 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.667015 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.667684 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.667685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.668976 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.668979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.669490 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.669491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.669698 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.670302 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:19.670313 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:19.672931 (+ 2618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
W20260806 06:36:19.673173 10696 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:19.665219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:47685 (local address 127.29.142.190:33211)
0806 06:36:19.665308 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:19.665311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:19.665520 (+ 209us) server_negotiation.cc:407] Connection header received
0806 06:36:19.666021 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:19.666022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:19.666033 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:19.666084 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:19.667095 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.667584 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.669062 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.669394 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.669465 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.673101 (+ 3636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:47685: BlockingRecv error: recv got EOF from 127.29.142.131:47685 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 06:36:19.673590 11063 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:33211 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.142.190:33211: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
I20260806 06:36:19.676416 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:19.668399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43448 (local address 127.29.142.190:33211)
0806 06:36:19.668558 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:19.668560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:19.668572 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:36:19.668591 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:19.668592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:19.668601 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:19.668720 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:19.669478 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.670207 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.671046 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.671257 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.671349 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.672810 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:19.672818 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:19.672818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:19.672833 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:19.674932 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:19.675324 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:19.675328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:19.675328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:19.675433 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:19.675658 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:19.675659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:19.675660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:19.675899 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:19.676051 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:19.676171 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 06:36:19.676289 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:19.678347 11066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:19.673856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:50771)
0806 06:36:19.673902 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:19.673918 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:19.673952 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:19.674337 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:19.674338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:19.674476 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:19.674527 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.674528 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.675228 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.675229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.675751 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.675752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.676106 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.676107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.676184 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.676614 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:19.676619 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:19.678300 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18}
W20260806 06:36:19.678449 10696 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:19.674161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:50771 (local address 127.29.142.190:33211)
0806 06:36:19.674223 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:19.674225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:19.674231 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:36:19.674249 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:19.674249 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:19.674256 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:19.674316 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:19.674607 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.675134 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.675841 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.676030 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.676108 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.678387 (+ 2279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:50771: BlockingRecv error: recv got EOF from 127.29.142.131:50771 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:19 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:19.682557 11066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:19.678695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:34685)
0806 06:36:19.678780 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:19.678791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:19.678840 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:19.678937 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:19.678938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:19.679045 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:19.679073 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.679074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.679636 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.679637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.680108 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:19.680109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.680420 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.680421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:19.680456 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.680889 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:19.680894 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:19.682508 (+ 1614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260806 06:36:19.682708 10696 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:19.678739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:34685 (local address 127.29.142.190:33211)
0806 06:36:19.678791 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:19.678793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:19.678827 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:36:19.678879 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:19.678880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:19.678887 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:19.678935 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:19.679130 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.679565 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.680181 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:19.680352 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:19.680403 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:19.682598 (+ 2195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:34685: BlockingRecv error: recv got EOF from 127.29.142.131:34685 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:36:19.682790 11063 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 06:36:20 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:20 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:20.407557 11070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:20.402558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:46433)
0806 06:36:20.402725 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:20.402740 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:20.402832 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:20.402985 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:20.402986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:20.403173 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:20.403227 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:20.403229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:20.404062 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:20.404063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:20.404620 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:20.404621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:20.404998 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:20.404998 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:20.405041 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:20.405541 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:20.405546 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:20.407483 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
W20260806 06:36:20.407721 11073 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:20.402669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:46433 (local address 127.29.142.190:33211)
0806 06:36:20.402831 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:20.402833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:20.402845 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:36:20.402892 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:20.402893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:20.402906 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:20.402962 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:20.403359 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:20.403978 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:20.404726 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:20.404926 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:20.404983 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:20.407619 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:46433: BlockingRecv error: recv got EOF from 127.29.142.129:46433 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
Aug 06 06:36:20 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:20 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:20.554229 11075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:20.546116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:39367)
0806 06:36:20.546312 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:20.546324 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:20.546393 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:20.546543 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:20.546544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:20.546756 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:20.546813 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:20.546815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:20.547499 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:20.547501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:20.548065 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:20.548067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:20.548444 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:20.548445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:20.548514 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:20.552034 (+ 3520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:20.552040 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:20.554141 (+ 2101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":122,"threads_started":1}
W20260806 06:36:20.554369 11073 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:20.546208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:39367 (local address 127.29.142.190:33211)
0806 06:36:20.546304 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:20.546306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:20.546394 (+ 88us) server_negotiation.cc:407] Connection header received
0806 06:36:20.546444 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:20.546444 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:20.546457 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:20.546522 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:20.546886 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:20.547392 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:20.548151 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:20.548347 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:20.548402 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:20.554275 (+ 5873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:39367: BlockingRecv error: recv got EOF from 127.29.142.130:39367 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 06:36:20 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:20 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:20.688625 11076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:20.683314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:50757)
0806 06:36:20.683546 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:20.683563 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:20.683636 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:20.683811 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:20.683811 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:20.684003 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:20.684059 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:20.684060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:20.684845 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:20.684846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:20.685576 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:20.685579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:20.685955 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:20.685957 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:20.686016 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:20.686538 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:20.686543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:20.688507 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":140,"threads_started":1}
W20260806 06:36:20.688666 11073 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:20.683437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:50757 (local address 127.29.142.190:33211)
0806 06:36:20.683542 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:20.683548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:20.683636 (+ 88us) server_negotiation.cc:407] Connection header received
0806 06:36:20.683719 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:20.683720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:20.683735 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:20.683788 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:20.684134 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:20.684718 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:20.685678 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:20.685874 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:20.685954 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:20.688597 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:50757: BlockingRecv error: recv got EOF from 127.29.142.131:50757 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 06:36:21 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:21 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:21.412720 11078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:21.408081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:42641)
0806 06:36:21.408287 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:21.408299 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:21.408365 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:21.408486 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:21.408486 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:21.408671 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:21.408724 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:21.408725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:21.409363 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:21.409364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:21.409900 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:21.409902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:21.410253 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:21.410254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:21.410306 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:21.410827 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:21.410832 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:21.412657 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":122,"threads_started":1}
W20260806 06:36:21.412886 11079 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:21.408176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:42641 (local address 127.29.142.190:33211)
0806 06:36:21.408290 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:21.408292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:21.408349 (+ 57us) server_negotiation.cc:407] Connection header received
0806 06:36:21.408411 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:21.408411 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:21.408422 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:21.408474 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:21.408786 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:21.409291 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:21.409976 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:21.410165 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:21.410216 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:21.412779 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:42641: BlockingRecv error: recv got EOF from 127.29.142.129:42641 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
Aug 06 06:36:21 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:21 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:21.559803 11080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:21.554895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:51455)
0806 06:36:21.555054 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:21.555081 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:21.555148 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:21.555292 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:21.555293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:21.555484 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:21.555537 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:21.555539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:21.556202 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:21.556204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:21.556798 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:21.556800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:21.557139 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:21.557140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:21.557187 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:21.557769 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:21.557774 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:21.559735 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
W20260806 06:36:21.559921 11079 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:21.555001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:51455 (local address 127.29.142.190:33211)
0806 06:36:21.555058 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:21.555060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:21.555147 (+ 87us) server_negotiation.cc:407] Connection header received
0806 06:36:21.555201 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:21.555202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:21.555214 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:21.555274 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:21.555611 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:21.556104 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:21.556884 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:21.557067 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:21.557118 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:21.559845 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:51455: BlockingRecv error: recv got EOF from 127.29.142.130:51455 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 06 06:36:21 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:21 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:21.694465 11081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:21.689356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:46377)
0806 06:36:21.689529 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:21.689542 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:21.689609 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:21.689751 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:21.689752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:21.689955 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:21.690009 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:21.690010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:21.690646 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:21.690647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:21.691262 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:21.691265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:21.691610 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:21.691611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:21.691691 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:21.692192 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:21.692197 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:21.694359 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
W20260806 06:36:21.694518 11079 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:21.689532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:46377 (local address 127.29.142.190:33211)
0806 06:36:21.689603 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:21.689605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:21.689616 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:21.689665 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:21.689665 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:21.689679 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:21.689731 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:21.690076 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:21.690570 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:21.691343 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:21.691540 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:21.691596 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:21.694452 (+ 2856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:46377: BlockingRecv error: recv got EOF from 127.29.142.131:46377 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 06:36:22 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:22 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:22.418776 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:22.413358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:34953)
0806 06:36:22.413548 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:22.413567 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:22.413662 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:22.414044 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:22.414046 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:22.414249 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:22.414308 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:22.414311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:22.415074 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:22.415075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:22.415618 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:22.415620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:22.415999 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:22.416000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:22.416063 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:22.416638 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:22.416644 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:22.418622 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":102,"threads_started":1}
W20260806 06:36:22.418983 11085 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:22.413531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:34953 (local address 127.29.142.190:33211)
0806 06:36:22.413678 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:22.413680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:22.413811 (+ 131us) server_negotiation.cc:407] Connection header received
0806 06:36:22.413927 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:22.413929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:22.413950 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:22.414034 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:22.414396 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:22.414978 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:22.415711 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:22.415918 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:22.415999 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:22.418883 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:34953: BlockingRecv error: recv got EOF from 127.29.142.129:34953 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
Aug 06 06:36:22 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:22 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:22.565487 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:22.560383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:52971)
0806 06:36:22.560581 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:22.560600 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:22.560687 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:22.560862 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:22.560863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:22.561063 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:22.561115 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:22.561117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:22.561841 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:22.561842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:22.562380 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:22.562382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:22.562865 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:22.562874 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:22.562937 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:22.563413 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:22.563417 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:22.565423 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":105,"threads_started":1}
W20260806 06:36:22.565627 11085 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:22.560479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:52971 (local address 127.29.142.190:33211)
0806 06:36:22.560562 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:22.560565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:22.560666 (+ 101us) server_negotiation.cc:407] Connection header received
0806 06:36:22.560784 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:22.560785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:22.560799 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:22.560843 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:22.561205 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:22.561741 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:22.562479 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:22.562775 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:22.562852 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:22.565542 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:52971: BlockingRecv error: recv got EOF from 127.29.142.130:52971 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 06:36:22 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:22 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:22.701469 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:22.694997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:48535)
0806 06:36:22.695152 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:22.695164 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:22.695246 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:22.695438 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:22.695439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:22.695656 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:22.695715 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:22.695716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:22.696720 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:22.696721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:22.697549 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:22.697552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:22.698103 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:22.698104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:22.698169 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:22.698838 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:22.698845 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:22.701381 (+ 2536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":107,"threads_started":1}
W20260806 06:36:22.701612 11085 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:22.695042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:48535 (local address 127.29.142.190:33211)
0806 06:36:22.695095 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:22.695097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:22.695218 (+ 121us) server_negotiation.cc:407] Connection header received
0806 06:36:22.695316 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:22.695317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:22.695331 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:22.695360 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:22.695805 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:22.696593 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:22.697663 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:22.697965 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:22.698009 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:22.701529 (+ 3520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:48535: BlockingRecv error: recv got EOF from 127.29.142.131:48535 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 06:36:23 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:23 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:23.424342 11089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:23.419329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:35841)
0806 06:36:23.419491 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:23.419510 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:23.419565 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:23.419799 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:23.419800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:23.419986 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:23.420040 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:23.420041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:23.420739 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:23.420740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:23.421283 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:23.421285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:23.421625 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:23.421626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:23.421667 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:23.422243 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:23.422248 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:23.424240 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
W20260806 06:36:23.424412 11090 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:23.419474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:35841 (local address 127.29.142.190:33211)
0806 06:36:23.419631 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:23.419635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:23.419657 (+ 22us) server_negotiation.cc:407] Connection header received
0806 06:36:23.419692 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:23.419693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:23.419704 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:23.419790 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:23.420138 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:23.420659 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:23.421360 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:23.421554 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:23.421609 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:23.424337 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:35841: BlockingRecv error: recv got EOF from 127.29.142.129:35841 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
Aug 06 06:36:23 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:23 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:23.571599 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:23.566088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:50935)
0806 06:36:23.566303 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:23.566318 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:23.566427 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:23.566779 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:23.566780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:23.566995 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:23.567057 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:23.567059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:23.567800 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:23.567801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:23.568414 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:23.568416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:23.568788 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:23.568789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:23.568855 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:23.569438 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:23.569443 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:23.571536 (+ 2093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":122,"threads_started":1}
W20260806 06:36:23.571717 11090 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:23.566220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:50935 (local address 127.29.142.190:33211)
0806 06:36:23.566308 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:23.566310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:23.566406 (+ 96us) server_negotiation.cc:407] Connection header received
0806 06:36:23.566629 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:23.566631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:23.566680 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:23.566754 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:23.567147 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:23.567703 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:23.568502 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:23.568710 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:23.568789 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:23.571639 (+ 2850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:50935: BlockingRecv error: recv got EOF from 127.29.142.130:50935 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 06:36:23 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:23 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:23.707427 11092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:23.702324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:58939)
0806 06:36:23.702517 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:23.702529 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:23.702623 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:23.702877 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:23.702878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:23.703086 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:23.703137 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:23.703139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:23.703831 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:23.703833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:23.704370 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:23.704372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:23.704732 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:23.704733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:23.704792 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:23.705333 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:23.705339 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:23.707357 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":102,"threads_started":1}
W20260806 06:36:23.707566 11090 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:23.702388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:58939 (local address 127.29.142.190:33211)
0806 06:36:23.702471 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:23.702473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:23.702604 (+ 131us) server_negotiation.cc:407] Connection header received
0806 06:36:23.702726 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:23.702727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:23.702764 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:23.702861 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:23.703223 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:23.703725 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:23.704459 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:23.704654 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:23.704733 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:23.707481 (+ 2748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:58939: BlockingRecv error: recv got EOF from 127.29.142.131:58939 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 06:36:24 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:24 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:24.430608 11093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:24.425042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:50123)
0806 06:36:24.425272 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:24.425286 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:24.425371 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:24.425587 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:24.425588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:24.425786 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:24.425839 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:24.425840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:24.426552 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:24.426553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:24.427284 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:24.427288 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:24.427844 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:24.427845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:24.427909 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:24.428530 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:24.428536 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:24.430505 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":135,"threads_started":1}
W20260806 06:36:24.430711 11094 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:24.425163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:50123 (local address 127.29.142.190:33211)
0806 06:36:24.425292 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:24.425298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:24.425382 (+ 84us) server_negotiation.cc:407] Connection header received
0806 06:36:24.425417 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:24.425418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:24.425429 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:24.425542 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:24.425913 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:24.426457 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:24.427415 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:24.427746 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:24.427817 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:24.430604 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:50123: BlockingRecv error: recv got EOF from 127.29.142.129:50123 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
Aug 06 06:36:24 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:24 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:24.577512 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:24.572234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:36455)
0806 06:36:24.572406 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:24.572437 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:24.572509 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:24.572743 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:24.572744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:24.572994 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:24.573055 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:24.573057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:24.573801 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:24.573802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:24.574351 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:24.574353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:24.574785 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:24.574786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:24.574843 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:24.575452 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:24.575458 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:24.577444 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
W20260806 06:36:24.577711 11094 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:24.572484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:36455 (local address 127.29.142.190:33211)
0806 06:36:24.572563 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:24.572566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:24.572577 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:24.572609 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:24.572610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:24.572624 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:24.572715 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:24.573147 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:24.573708 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:24.574442 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:24.574636 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:24.574766 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:24.577648 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:36455: BlockingRecv error: recv got EOF from 127.29.142.130:36455 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 06:36:24 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:24 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:24.713478 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:24.707986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:38113)
0806 06:36:24.708329 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:24.708343 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:24.708451 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:24.708615 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:24.708616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:24.708845 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:24.708901 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:24.708903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:24.709683 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:24.709684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:24.710280 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:24.710281 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:24.710780 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:24.710781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:24.710853 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:24.711461 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:24.711467 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:24.713418 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":219,"threads_started":1}
W20260806 06:36:24.713645 11094 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:24.708208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:38113 (local address 127.29.142.190:33211)
0806 06:36:24.708308 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:24.708311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:24.708426 (+ 115us) server_negotiation.cc:407] Connection header received
0806 06:36:24.708527 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:24.708528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:24.708545 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:24.708607 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:24.708999 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:24.709571 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:24.710376 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:24.710624 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:24.710778 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:24.713556 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:38113: BlockingRecv error: recv got EOF from 127.29.142.131:38113 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 06:36:25 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:25 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:25.436094 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:25.431181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:60611)
0806 06:36:25.431365 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:25.431377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:25.431446 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:25.431603 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:25.431604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:25.431813 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:25.431865 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:25.431867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:25.432532 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:25.432532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:25.433074 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:25.433075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:25.433443 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:25.433443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:25.433485 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:25.433971 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:25.433975 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:25.436030 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":105,"threads_started":1}
W20260806 06:36:25.436256 11098 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:25.431282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:60611 (local address 127.29.142.190:33211)
0806 06:36:25.431409 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:25.431411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:25.431425 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:36:25.431508 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:25.431509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:25.431520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:25.431586 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:25.431932 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:25.432454 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:25.433148 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:25.433359 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:25.433418 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:25.436159 (+ 2741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:60611: BlockingRecv error: recv got EOF from 127.29.142.129:60611 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
Aug 06 06:36:25 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:25 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:25.588296 11100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:25.578108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:52683)
0806 06:36:25.578291 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:25.578306 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:25.578403 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:25.578569 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:25.578570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:25.578788 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:25.578839 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:25.578842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:25.579596 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:25.579598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:25.580126 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:25.580127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:25.580511 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:25.580512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:25.580568 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:25.586262 (+ 5694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:25.586268 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:25.588210 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
W20260806 06:36:25.588428 11098 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:25.578211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:52683 (local address 127.29.142.190:33211)
0806 06:36:25.578295 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:25.578298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:25.578382 (+ 84us) server_negotiation.cc:407] Connection header received
0806 06:36:25.578486 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:25.578487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:25.578500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:25.578551 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:25.578943 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:25.579500 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:25.580216 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:25.580431 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:25.580514 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:25.588344 (+ 7830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:52683: BlockingRecv error: recv got EOF from 127.29.142.130:52683 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 06:36:25 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:25 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:25.719089 11102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:25.714014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:35057)
0806 06:36:25.714246 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:25.714258 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:25.714362 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:25.714527 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:25.714529 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:25.714774 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:25.714825 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:25.714827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:25.715567 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:25.715568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:25.716104 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:25.716105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:25.716484 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:25.716485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:25.716543 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:25.717053 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:25.717058 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:25.719025 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":127,"threads_started":1}
W20260806 06:36:25.719219 11098 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:25.714137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:35057 (local address 127.29.142.190:33211)
0806 06:36:25.714222 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:25.714225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:25.714340 (+ 115us) server_negotiation.cc:407] Connection header received
0806 06:36:25.714435 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:25.714436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:25.714456 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:25.714521 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:25.714922 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:25.715462 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:25.716196 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:25.716393 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:25.716482 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:25.719134 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:35057: BlockingRecv error: recv got EOF from 127.29.142.131:35057 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 06:36:26 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:26 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:26.441373 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:26.436586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:37237)
0806 06:36:26.436780 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:26.436793 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:26.436866 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:26.437019 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:26.437020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:26.437199 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:26.437252 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:26.437254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:26.437903 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:26.437904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:26.438427 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:26.438428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:26.438797 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:26.438799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:26.438850 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:26.439335 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:26.439340 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:26.441311 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
W20260806 06:36:26.441512 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:26.436687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:37237 (local address 127.29.142.190:33211)
0806 06:36:26.436802 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:26.436804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:26.436846 (+ 42us) server_negotiation.cc:407] Connection header received
0806 06:36:26.436926 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:26.436926 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:26.436937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:26.436997 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:26.437316 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:26.437831 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:26.438499 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:26.438718 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:26.438769 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:26.441420 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:37237: BlockingRecv error: recv got EOF from 127.29.142.129:37237 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":57,"threads_started":1}
Aug 06 06:36:26 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:26 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:26.593995 11106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:26.589038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:37987)
0806 06:36:26.589211 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:26.589226 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:26.589312 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:26.589497 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:26.589498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:26.589752 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:26.589820 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:26.589823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:26.590517 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:26.590518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:26.591073 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:26.591075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:26.591421 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:26.591421 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:26.591461 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:26.592030 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:26.592036 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:26.593932 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
W20260806 06:36:26.594105 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:26.589101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:37987 (local address 127.29.142.190:33211)
0806 06:36:26.589210 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:26.589213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:26.589323 (+ 110us) server_negotiation.cc:407] Connection header received
0806 06:36:26.589367 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:26.589368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:26.589383 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:26.589463 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:26.589908 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:26.590438 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:26.591149 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:26.591349 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:26.591407 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:26.594035 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:37987: BlockingRecv error: recv got EOF from 127.29.142.130:37987 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 06:36:26 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:26 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:26.724637 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:26.719717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:54167)
0806 06:36:26.719876 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:26.719888 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:26.719954 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:26.720110 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:26.720111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:26.720343 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:26.720398 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:26.720400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:26.721059 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:26.721060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:26.721623 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:26.721626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:26.722034 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:26.722036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:26.722101 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:26.722610 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:26.722614 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:26.724534 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
W20260806 06:36:26.724735 11105 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:26.719819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:54167 (local address 127.29.142.190:33211)
0806 06:36:26.719913 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:26.719916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:26.719936 (+ 20us) server_negotiation.cc:407] Connection header received
0806 06:36:26.720018 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:26.720019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:26.720033 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:26.720095 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:26.720471 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:26.720979 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:26.721741 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:26.721943 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:26.722033 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:26.724627 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:54167: BlockingRecv error: recv got EOF from 127.29.142.131:54167 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 06:36:27 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:27 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:27.446913 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:27.441927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:43435)
0806 06:36:27.442104 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:27.442115 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:27.442166 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:27.442423 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:27.442424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:27.442597 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:27.442700 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:27.442702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:27.443413 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:27.443414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:27.443935 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:27.443936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:27.444279 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:27.444280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:27.444325 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:27.444856 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:27.444861 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:27.446851 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
W20260806 06:36:27.447036 11110 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:27.442055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:43435 (local address 127.29.142.190:33211)
0806 06:36:27.442299 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:27.442301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:27.442309 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:36:27.442332 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:27.442332 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:27.442342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:27.442406 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:27.442825 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:27.443340 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:27.444015 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:27.444201 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:27.444264 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:27.446954 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:43435: BlockingRecv error: recv got EOF from 127.29.142.129:43435 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":185,"threads_started":1}
Aug 06 06:36:27 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:27 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:27.600677 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:27.595029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:37973)
0806 06:36:27.595224 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:27.595237 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:27.595309 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:27.595479 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:27.595480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:27.595663 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:27.595717 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:27.595719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:27.596858 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:27.596859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:27.597397 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:27.597398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:27.597816 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:27.597816 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:27.597856 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:27.598394 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:27.598399 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:27.600560 (+ 2161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":125,"threads_started":1}
W20260806 06:36:27.600768 11110 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:27.595145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:37973 (local address 127.29.142.190:33211)
0806 06:36:27.595272 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:27.595274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:27.595290 (+ 16us) server_negotiation.cc:407] Connection header received
0806 06:36:27.595370 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:27.595371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:27.595384 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:27.595458 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:27.595812 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:27.596780 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:27.597496 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:27.597740 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:27.597807 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:27.600658 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:37973: BlockingRecv error: recv got EOF from 127.29.142.130:37973 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 06 06:36:27 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:27 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:27.730553 11114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:27.725374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:44613)
0806 06:36:27.725561 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:27.725573 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:27.725680 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:27.725840 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:27.725840 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:27.726024 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:27.726077 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:27.726079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:27.726770 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:27.726773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:27.727318 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:27.727320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:27.727726 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:27.727727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:27.727779 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:27.728467 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:27.728472 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:27.730479 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":109,"threads_started":1}
W20260806 06:36:27.730657 11110 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:27.725436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:44613 (local address 127.29.142.190:33211)
0806 06:36:27.725515 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:27.725518 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:27.725650 (+ 132us) server_negotiation.cc:407] Connection header received
0806 06:36:27.725758 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:27.725761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:27.725780 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:27.725844 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:27.726152 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:27.726685 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:27.727409 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:27.727599 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:27.727657 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:27.730587 (+ 2930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:44613: BlockingRecv error: recv got EOF from 127.29.142.131:44613 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 06:36:28 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:28 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:28.452932 11115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:28.447588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:32903)
0806 06:36:28.447771 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:28.447783 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:28.447854 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:28.448024 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:28.448025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:28.448214 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:28.448266 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:28.448268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:28.448981 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:28.448982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:28.449525 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:28.449527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:28.450073 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:28.450075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:28.450143 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:28.450725 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:28.450730 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:28.452859 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":108,"threads_started":1}
W20260806 06:36:28.453104 11116 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:28.447698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:32903 (local address 127.29.142.190:33211)
0806 06:36:28.447867 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:28.447869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:28.447899 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:36:28.447930 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:28.447931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:28.447942 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:28.447998 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:28.448337 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:28.448902 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:28.449586 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:28.449960 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:28.450041 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:28.453002 (+ 2961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:32903: BlockingRecv error: recv got EOF from 127.29.142.129:32903 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
Aug 06 06:36:28 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:28 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:28.606760 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:28.601438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:57385)
0806 06:36:28.601617 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:28.601629 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:28.601705 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:28.601881 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:28.601882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:28.602074 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:28.602128 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:28.602130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:28.602918 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:28.602919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:28.603483 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:28.603486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:28.603828 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:28.603829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:28.603868 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:28.604429 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:28.604434 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:28.606625 (+ 2191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
W20260806 06:36:28.606866 11116 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:28.601551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:57385 (local address 127.29.142.190:33211)
0806 06:36:28.601630 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:28.601632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:28.601759 (+ 127us) server_negotiation.cc:407] Connection header received
0806 06:36:28.601798 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:28.601799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:28.601813 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:28.601865 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:28.602197 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:28.602843 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:28.603579 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:28.603765 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:28.603820 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:28.606770 (+ 2950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:57385: BlockingRecv error: recv got EOF from 127.29.142.130:57385 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 06:36:28 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:28 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:28.736063 11118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:28.731120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:43943)
0806 06:36:28.731286 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:28.731298 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:28.731373 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:28.731522 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:28.731523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:28.731717 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:28.731774 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:28.731776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:28.732410 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:28.732411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:28.732935 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:28.732937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:28.733271 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:28.733273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:28.733326 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:28.733831 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:28.733836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:28.735959 (+ 2123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
W20260806 06:36:28.736132 11116 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:28.731216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:43943 (local address 127.29.142.190:33211)
0806 06:36:28.731311 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:28.731313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:28.731353 (+ 40us) server_negotiation.cc:407] Connection header received
0806 06:36:28.731432 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:28.731433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:28.731447 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:28.731498 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:28.731841 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:28.732341 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:28.733008 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:28.733185 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:28.733239 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:28.736064 (+ 2825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:43943: BlockingRecv error: recv got EOF from 127.29.142.131:43943 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 06:36:29 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:29 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:29.459007 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:29.453659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:53311)
0806 06:36:29.453830 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:29.453843 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:29.453917 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:29.454112 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:29.454114 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:29.454310 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:29.454364 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:29.454366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:29.455164 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:29.455166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:29.455726 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:29.455728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:29.456136 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:29.456138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:29.456199 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:29.456787 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:29.456793 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:29.458940 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":150,"threads_started":1}
W20260806 06:36:29.459200 11121 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:29.453791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:53311 (local address 127.29.142.190:33211)
0806 06:36:29.453926 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:29.453928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:29.453939 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:29.453990 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:29.453991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:29.454003 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:29.454060 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:29.454478 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:29.455048 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:29.455830 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:29.456038 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:29.456099 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:29.459083 (+ 2984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:53311: BlockingRecv error: recv got EOF from 127.29.142.129:53311 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
Aug 06 06:36:29 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:29 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:29.612138 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:29.607339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:43113)
0806 06:36:29.607522 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:29.607538 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:29.607606 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:29.607804 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:29.607805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:29.607985 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:29.608038 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:29.608040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:29.608711 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:29.608712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:29.609250 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:29.609251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:29.609637 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:29.609638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:29.609678 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:29.610193 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:29.610197 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:29.612074 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":118,"threads_started":1}
W20260806 06:36:29.612289 11121 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:29.607472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:43113 (local address 127.29.142.190:33211)
0806 06:36:29.607549 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:29.607551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:29.607608 (+ 57us) server_negotiation.cc:407] Connection header received
0806 06:36:29.607714 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:29.607715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:29.607728 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:29.607777 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:29.608111 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:29.608610 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:29.609367 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:29.609563 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:29.609628 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:29.612198 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:43113: BlockingRecv error: recv got EOF from 127.29.142.130:43113 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 06:36:29 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:29 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:29.742419 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:29.736788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:53649)
0806 06:36:29.736994 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:29.737008 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:29.737091 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:29.737333 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:29.737335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:29.737556 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:29.737608 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:29.737610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:29.738395 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:29.738396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:29.739038 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:29.739039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:29.739477 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:29.739479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:29.739555 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:29.740114 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:29.740120 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:29.742304 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":89,"threads_started":1}
W20260806 06:36:29.742483 11121 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:29.736902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:53649 (local address 127.29.142.190:33211)
0806 06:36:29.736995 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:29.736998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:29.737067 (+ 69us) server_negotiation.cc:407] Connection header received
0806 06:36:29.737208 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:29.737209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:29.737234 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:29.737323 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:29.737731 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:29.738289 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:29.739153 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:29.739372 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:29.739464 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:29.742410 (+ 2946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:53649: BlockingRecv error: recv got EOF from 127.29.142.131:53649 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 06:36:30 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:30 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:30.464785 11131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:30.459707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:37661)
0806 06:36:30.459903 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:30.459925 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:30.459987 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:30.460169 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:30.460170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:30.460373 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:30.460425 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:30.460427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:30.461116 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:30.461117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:30.461681 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:30.461683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:30.462035 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:30.462036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:30.462075 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:30.462561 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:30.462566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:30.464707 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":133,"threads_started":1}
W20260806 06:36:30.464931 11132 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:30.459856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:37661 (local address 127.29.142.190:33211)
0806 06:36:30.460001 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:30.460007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:30.460045 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:36:30.460075 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:30.460076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:30.460089 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:30.460147 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:30.460506 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:30.461025 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:30.461768 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:30.461964 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:30.462023 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:30.464838 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:37661: BlockingRecv error: recv got EOF from 127.29.142.129:37661 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
Aug 06 06:36:30 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:30 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:30.625656 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:30.612771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:37983)
0806 06:36:30.613007 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:30.613020 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:30.613106 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:30.613279 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:30.613280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:30.613498 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:30.613550 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:30.613552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:30.614274 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:30.614276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:30.614939 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:30.614941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:30.615314 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:30.615315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:30.615370 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:30.623248 (+ 7878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:30.623256 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:30.625538 (+ 2282us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":135,"threads_started":1}
W20260806 06:36:30.625780 11132 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:30.612898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:37983 (local address 127.29.142.190:33211)
0806 06:36:30.613001 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:30.613004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:30.613084 (+ 80us) server_negotiation.cc:407] Connection header received
0806 06:36:30.613173 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:30.613174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:30.613188 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:30.613273 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:30.613648 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:30.614170 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:30.615033 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:30.615235 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:30.615317 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:30.625701 (+ 10384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:37983: BlockingRecv error: recv got EOF from 127.29.142.130:37983 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 06:36:30 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:30 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:30.747833 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:30.742930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:42249)
0806 06:36:30.743085 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:30.743096 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:30.743158 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:30.743316 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:30.743317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:30.743504 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:30.743554 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:30.743555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:30.744289 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:30.744290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:30.744836 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:30.744837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:30.745191 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:30.745191 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:30.745233 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:30.745763 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:30.745768 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:30.747743 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
W20260806 06:36:30.747946 11132 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:30.743023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:42249 (local address 127.29.142.190:33211)
0806 06:36:30.743095 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:30.743097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:30.743158 (+ 61us) server_negotiation.cc:407] Connection header received
0806 06:36:30.743220 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:30.743220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:30.743234 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:30.743280 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:30.743654 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:30.744188 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:30.744921 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:30.745117 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:30.745176 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:30.747850 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:42249: BlockingRecv error: recv got EOF from 127.29.142.131:42249 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 06:36:31 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:31 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:31.470520 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:31.465295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:41065)
0806 06:36:31.465452 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:31.465465 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:31.465529 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:31.465721 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:31.465722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:31.465914 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:31.465966 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:31.465968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:31.466724 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:31.466725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:31.467311 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:31.467313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:31.467684 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:31.467685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:31.467737 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:31.468345 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:31.468350 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:31.470457 (+ 2107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":91,"threads_started":1}
W20260806 06:36:31.470727 11145 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:31.465425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:41065 (local address 127.29.142.190:33211)
0806 06:36:31.465558 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:31.465560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:31.465569 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:36:31.465603 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:31.465604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:31.465614 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:31.465686 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:31.466052 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:31.466583 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:31.467389 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:31.467582 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:31.467638 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:31.470598 (+ 2960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:41065: BlockingRecv error: recv got EOF from 127.29.142.129:41065 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
Aug 06 06:36:31 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:31 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:31.631273 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:31.626209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:47953)
0806 06:36:31.626368 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:31.626381 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:31.626457 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:31.626603 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:31.626604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:31.626838 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:31.626890 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:31.626892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:31.627603 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:31.627605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:31.628131 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:31.628133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:31.628516 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:31.628517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:31.628581 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:31.629132 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:31.629137 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:31.631181 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
W20260806 06:36:31.631405 11145 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:31.626307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:47953 (local address 127.29.142.190:33211)
0806 06:36:31.626380 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:31.626382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:31.626461 (+ 79us) server_negotiation.cc:407] Connection header received
0806 06:36:31.626525 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:31.626526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:31.626537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:31.626581 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:31.627006 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:31.627514 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:31.628223 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:31.628427 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:31.628504 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:31.631298 (+ 2794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:47953: BlockingRecv error: recv got EOF from 127.29.142.130:47953 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 06:36:31 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:31 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:31.754109 11147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:31.748498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:59459)
0806 06:36:31.748710 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:31.748722 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:31.748797 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:31.748946 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:31.748946 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:31.749134 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:31.749183 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:31.749185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:31.749890 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:31.749892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:31.750466 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:31.750468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:31.750929 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:31.750931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:31.751004 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:31.751620 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:31.751627 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:31.754045 (+ 2418us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":139,"threads_started":1}
W20260806 06:36:31.754248 11145 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:31.748646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:59459 (local address 127.29.142.190:33211)
0806 06:36:31.748715 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:31.748719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:31.748799 (+ 80us) server_negotiation.cc:407] Connection header received
0806 06:36:31.748865 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:31.748866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:31.748878 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:31.748923 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:31.749262 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:31.749776 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:31.750566 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:31.750835 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:31.750926 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:31.754152 (+ 3226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:59459: BlockingRecv error: recv got EOF from 127.29.142.131:59459 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 06:36:32 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:32 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:32.475958 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:32.471062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:36139)
0806 06:36:32.471262 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:32.471276 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:32.471346 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:32.471519 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:32.471521 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:32.471742 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:32.471793 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:32.471795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:32.472458 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:32.472458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:32.472994 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:32.472995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:32.473331 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:32.473332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:32.473372 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:32.473878 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:32.473882 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:32.475894 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":121,"threads_started":1}
W20260806 06:36:32.476115 11150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:32.471093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:36139 (local address 127.29.142.190:33211)
0806 06:36:32.471277 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:32.471281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:32.471325 (+ 44us) server_negotiation.cc:407] Connection header received
0806 06:36:32.471414 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:32.471415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:32.471429 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:32.471499 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:32.471864 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:32.472376 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:32.473071 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:32.473265 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:32.473317 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:32.476024 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:36139: BlockingRecv error: recv got EOF from 127.29.142.129:36139 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":79,"threads_started":1}
Aug 06 06:36:32 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:32 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:32.636900 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:32.631856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:53679)
0806 06:36:32.632029 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:32.632042 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:32.632101 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:32.632285 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:32.632286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:32.632473 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:32.632527 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:32.632529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:32.633224 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:32.633225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:32.633782 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:32.633783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:32.634135 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:32.634135 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:32.634178 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:32.634824 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:32.634829 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:32.636801 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
W20260806 06:36:32.636973 11150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:32.632010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:53679 (local address 127.29.142.190:33211)
0806 06:36:32.632084 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:32.632086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:32.632097 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:32.632184 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:32.632185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:32.632197 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:32.632251 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:32.632604 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:32.633145 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:32.633863 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:32.634060 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:32.634119 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:32.636906 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:53679: BlockingRecv error: recv got EOF from 127.29.142.130:53679 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 06:36:32 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:32 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:32.760303 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:32.754767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:59997)
0806 06:36:32.754972 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:32.754987 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:32.755074 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:32.755249 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:32.755250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:32.755537 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:32.755601 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:32.755604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:32.756311 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:32.756312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:32.756877 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:32.756878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:32.757212 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:32.757213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:32.757253 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:32.757966 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:32.757972 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:32.760229 (+ 2257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":115,"threads_started":1}
W20260806 06:36:32.760438 11150 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:32.754927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:59997 (local address 127.29.142.190:33211)
0806 06:36:32.755002 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:32.755004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:32.755054 (+ 50us) server_negotiation.cc:407] Connection header received
0806 06:36:32.755127 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:32.755128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:32.755141 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:32.755200 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:32.755702 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:32.756232 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:32.756956 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:32.757136 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:32.757189 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:32.760345 (+ 3156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:59997: BlockingRecv error: recv got EOF from 127.29.142.131:59997 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 06:36:33 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:33 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:33.481765 11157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:33.476517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:44517)
0806 06:36:33.476701 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:33.476716 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:33.476798 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:33.477005 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:33.477005 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:33.477193 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:33.477251 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:33.477253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:33.478216 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:33.478217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:33.478895 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:33.478898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:33.479266 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:33.479267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:33.479306 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:33.479850 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:33.479854 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:33.481689 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":78,"threads_started":1}
W20260806 06:36:33.481877 11158 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:33.476652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:44517 (local address 127.29.142.190:33211)
0806 06:36:33.476860 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:33.476863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:33.476873 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:36:33.476901 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:33.476902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:33.476913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:33.476978 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:33.477329 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:33.478116 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:33.478979 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:33.479186 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:33.479244 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:33.481799 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:44517: BlockingRecv error: recv got EOF from 127.29.142.129:44517 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":126,"threads_started":1}
Aug 06 06:36:33 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:33 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:33.642717 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:33.637478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:48981)
0806 06:36:33.637654 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:33.637667 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:33.637737 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:33.637913 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:33.637914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:33.638101 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:33.638177 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:33.638179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:33.639089 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:33.639089 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:33.639616 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:33.639617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:33.639965 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:33.639966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:33.640016 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:33.640496 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:33.640500 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:33.642562 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":119,"threads_started":1}
W20260806 06:36:33.642997 11158 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:33.637540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:48981 (local address 127.29.142.190:33211)
0806 06:36:33.637611 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:33.637613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:33.637734 (+ 121us) server_negotiation.cc:407] Connection header received
0806 06:36:33.637820 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:33.637821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:33.637833 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:33.637891 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:33.638240 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:33.639016 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:33.639694 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:33.639892 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:33.639948 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:33.642736 (+ 2788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:48981: BlockingRecv error: recv got EOF from 127.29.142.130:48981 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 06:36:33 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:33 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.131@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:33.766428 11160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:33.760950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.131:52479)
0806 06:36:33.761151 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:33.761166 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:33.761290 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:33.761479 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:33.761480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:33.761686 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:33.761739 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:33.761741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:33.762454 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:33.762456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:33.763043 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:33.763049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:33.763467 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:33.763468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:33.763535 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:33.764102 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:33.764107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:33.766333 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":97,"threads_started":1}
W20260806 06:36:33.766566 11158 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:33.761076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:52479 (local address 127.29.142.190:33211)
0806 06:36:33.761159 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:33.761162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:33.761264 (+ 102us) server_negotiation.cc:407] Connection header received
0806 06:36:33.761372 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:33.761373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:33.761390 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:33.761470 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:33.761829 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:33.762345 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:33.763154 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:33.763354 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:33.763466 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:33.766493 (+ 3027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.131:52479: BlockingRecv error: recv got EOF from 127.29.142.131:52479 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 06:36:34 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:34 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.129@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:34.487629 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:34.482459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.129:41993)
0806 06:36:34.482621 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:34.482641 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:34.482750 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:34.482932 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:34.482933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:34.483119 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:34.483172 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:34.483174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:34.483881 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:34.483883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:34.484582 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:34.484585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:34.485087 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:34.485088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:34.485141 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:34.485639 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:34.485644 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:34.487564 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
W20260806 06:36:34.487747 11162 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:34.482602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:41993 (local address 127.29.142.190:33211)
0806 06:36:34.482782 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:34.482784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:34.482794 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:36:34.482824 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:34.482825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:34.482836 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:34.482913 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:34.483246 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:34.483780 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:34.484723 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:34.484997 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:34.485060 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:34.487670 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.129:41993: BlockingRecv error: recv got EOF from 127.29.142.129:41993 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":89,"threads_started":1}
Aug 06 06:36:34 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:36:34 dist-test-slave-3kk6 krb5kdc[9641](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.142.130@KRBTEST.COM for oryx/127.29.142.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:36:34.648538 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:34.643428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33211 (local address 127.29.142.130:55715)
0806 06:36:34.643634 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:34.643648 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:34.643753 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:34.644025 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:34.644027 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:34.644214 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:34.644262 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:34.644264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:34.644978 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:34.644980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:34.645511 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:34.645512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:34.645889 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:34.645890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:34.645956 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:34.646449 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:34.646453 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:34.648434 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.142.190:33211: Server oryx/127.29.142.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":121,"threads_started":1}
W20260806 06:36:34.648806 11162 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:34.643555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:55715 (local address 127.29.142.190:33211)
0806 06:36:34.643636 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:34.643638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:34.643891 (+ 253us) server_negotiation.cc:407] Connection header received
0806 06:36:34.643927 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:34.643927 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:34.643939 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:34.644015 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:34.644370 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:34.644888 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:34.645613 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:34.645813 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:34.645890 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:34.648572 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.142.130:55715: BlockingRecv error: recv got EOF from 127.29.142.130:55715 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:36:34.670099 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 10487
I20260806 06:36:34.683048 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 10697
I20260806 06:36:34.694000 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 10886
I20260806 06:36:34.702124 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 10391
2026-08-06T06:36:34Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17745 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T06:36:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:34Z Disabled control of system clock
I20260806 06:36:34.738535 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:37129
--webserver_interface=127.29.142.190
--webserver_port=0
--builtin_ntp_servers=127.29.142.148:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:37129
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 06:36:34.823670 11169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:34.823912 11169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:34.823935 11169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:34.825059 11169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:34.825096 11169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:34.825109 11169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:34.825121 11169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:34.825134 11169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:34.826428 11169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:37129
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:37129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:34.826633 11169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:34.826911 11169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:34.832353 11177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:34.832329 11178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:34.832329 11181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:34.832360 11169 server_base.cc:1034] running on GCE node
I20260806 06:36:34.832907 11169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:34.833254 11169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:34.834427 11169 hybrid_clock.cc:648] HybridClock initialized: now 1785998194834413 us; error 48 us; skew 500 ppm
I20260806 06:36:34.835902 11169 webserver.cc:460] Webserver started at http://127.29.142.190:44051/ using document root <none> and password file <none>
I20260806 06:36:34.836197 11169 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:34.836289 11169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:34.836457 11169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:34.837622 11169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "15ac623257c14d5da4820020f4b3e892"
format_stamp: "Formatted at 2026-08-06 06:36:34 on dist-test-slave-3kk6"
server_key: "7b9ef8189773e6fb4693870a3f23d064"
server_key_iv: "25e932281cbde69a56433e6357ccc5ed"
server_key_version: "encryptionkey@0"
I20260806 06:36:34.838023 11169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "15ac623257c14d5da4820020f4b3e892"
format_stamp: "Formatted at 2026-08-06 06:36:34 on dist-test-slave-3kk6"
server_key: "7b9ef8189773e6fb4693870a3f23d064"
server_key_iv: "25e932281cbde69a56433e6357ccc5ed"
server_key_version: "encryptionkey@0"
I20260806 06:36:34.839576 11169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:36:34.840381 11191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:34.840561 11169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:36:34.840677 11169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "15ac623257c14d5da4820020f4b3e892"
format_stamp: "Formatted at 2026-08-06 06:36:34 on dist-test-slave-3kk6"
server_key: "7b9ef8189773e6fb4693870a3f23d064"
server_key_iv: "25e932281cbde69a56433e6357ccc5ed"
server_key_version: "encryptionkey@0"
I20260806 06:36:34.840766 11169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:34.851280 11169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:34.851822 11169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:34.851949 11169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:34.856060 11169 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:37129
I20260806 06:36:34.856112 11257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:37129 every 8 connection(s)
I20260806 06:36:34.856495 11169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:34.857275 11259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:34.860028 11259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892: Bootstrap starting.
I20260806 06:36:34.860687 11259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:34.861021 11259 log.cc:824] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:34.861943 11259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892: No bootstrap required, opened a new log
I20260806 06:36:34.863690 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 11169
I20260806 06:36:34.863804 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:34.863849 11259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ac623257c14d5da4820020f4b3e892" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37129 } }
I20260806 06:36:34.864013 30266 external_mini_cluster.cc:1428] Setting key 51b4d232bd59ccd16cb9ad201509fa4e
I20260806 06:36:34.864042 11259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:34.864097 11259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15ac623257c14d5da4820020f4b3e892, State: Initialized, Role: FOLLOWER
I20260806 06:36:34.864257 11259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ac623257c14d5da4820020f4b3e892" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37129 } }
I20260806 06:36:34.864357 11259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:34.864395 11259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:34.864455 11259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:34.865206 11259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ac623257c14d5da4820020f4b3e892" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37129 } }
I20260806 06:36:34.865320 11259 leader_election.cc:302] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15ac623257c14d5da4820020f4b3e892; no voters:
I20260806 06:36:34.865483 11259 leader_election.cc:288] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:34.865547 11265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:34.865792 11265 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [term 1 LEADER]: Becoming Leader. State: Replica: 15ac623257c14d5da4820020f4b3e892, State: Running, Role: LEADER
I20260806 06:36:34.865876 11259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:34.865962 11265 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ac623257c14d5da4820020f4b3e892" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37129 } }
I20260806 06:36:34.866449 11267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15ac623257c14d5da4820020f4b3e892. Latest consensus state: current_term: 1 leader_uuid: "15ac623257c14d5da4820020f4b3e892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ac623257c14d5da4820020f4b3e892" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37129 } } }
I20260806 06:36:34.866525 11267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:34.866765 11274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:34.866879 11266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15ac623257c14d5da4820020f4b3e892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ac623257c14d5da4820020f4b3e892" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37129 } } }
I20260806 06:36:34.866943 11266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:34.867336 11274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:34.869050 11274 catalog_manager.cc:1347] Generated new cluster ID: fd64bf2f3bca4d4096f5a761eac9be10
I20260806 06:36:34.869107 11274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:36:34.870983 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:34.864829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60764 (local address 127.29.142.190:37129)
0806 06:36:34.865090 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:34.865101 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:34.865138 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:36:34.865262 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:34.865270 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:34.865379 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:34.865546 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:34.865825 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:34.867570 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:34.868651 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:34.869290 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:34.869420 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:34.870191 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:34.870196 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:34.870196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:34.870437 (+ 241us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:34.870480 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:34.870495 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:34.870571 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:34.870680 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 06:36:34.870766 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":111,"threads_started":1}
I20260806 06:36:34.900277 11274 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:34.900890 11274 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:34.905993 11274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15ac623257c14d5da4820020f4b3e892: Generated new TSK 0
I20260806 06:36:34.906234 11274 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:36:34.908586 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37129
--builtin_ntp_servers=127.29.142.148:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 06:36:34.996730 11290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:34.996978 11290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:34.997001 11290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:34.998193 11290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:34.998241 11290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:34.998267 11290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:36:34.999693 11290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:34.999979 11290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:35.000330 11290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:35.005844 11299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:35.005797 11300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:35.005828 11290 server_base.cc:1034] running on GCE node
W20260806 06:36:35.005823 11303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:35.006237 11290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:35.006531 11290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:35.007690 11290 hybrid_clock.cc:648] HybridClock initialized: now 1785998195007674 us; error 40 us; skew 500 ppm
I20260806 06:36:35.009083 11290 webserver.cc:460] Webserver started at http://127.29.142.129:34495/ using document root <none> and password file <none>
I20260806 06:36:35.009349 11290 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:35.009425 11290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:35.009579 11290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:35.010802 11290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "fc04226f666a4fbe8814e0114a992bb4"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "5ad9a4f6e151784cb96ea0e3a493b503"
server_key_iv: "c374058b537efdf6738771941510ef11"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.011185 11290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "fc04226f666a4fbe8814e0114a992bb4"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "5ad9a4f6e151784cb96ea0e3a493b503"
server_key_iv: "c374058b537efdf6738771941510ef11"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.012593 11290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:36:35.013370 11315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.013552 11290 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:36:35.013664 11290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "fc04226f666a4fbe8814e0114a992bb4"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "5ad9a4f6e151784cb96ea0e3a493b503"
server_key_iv: "c374058b537efdf6738771941510ef11"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.013759 11290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:35.065287 11290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:35.065644 11290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:35.065850 11290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:35.066144 11290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:35.066519 11290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:35.066577 11290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.066614 11290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:35.066699 11290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.072924 11290 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:33727
I20260806 06:36:35.072971 11481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:33727 every 8 connection(s)
I20260806 06:36:35.073344 11290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:36:35.075131 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 11290
I20260806 06:36:35.075273 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:36:35.075461 30266 external_mini_cluster.cc:1428] Setting key 70f38edccb7b526693448ac98eb99f29
I20260806 06:36:35.076562 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37129
--builtin_ntp_servers=127.29.142.148:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 06:36:35.079298 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.074157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:41459 (local address 127.29.142.190:37129)
0806 06:36:35.074223 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:35.074224 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:35.074497 (+ 273us) server_negotiation.cc:407] Connection header received
0806 06:36:35.075145 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:35.075147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:35.075162 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:35.075229 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:35.076178 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:35.076730 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:35.077737 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:35.077954 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:35.078035 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:35.078996 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:35.078998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:35.078998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:35.079003 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:35.079017 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:35.079021 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:35.079093 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:35.079176 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 06:36:35.079208 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:36:35.079519 11485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.074136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37129 (local address 127.29.142.129:41459)
0806 06:36:35.074367 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:35.074402 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:35.075026 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:35.075316 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:35.075319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:35.075346 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:35.076092 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:35.076099 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:35.076831 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:35.076832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:35.077634 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:35.077636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:35.078043 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:35.078044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:35.078192 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:35.078834 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:35.078835 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:35.078864 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:35.078887 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:35.079097 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:35.079103 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:35.079106 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:36:35.079169 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:36:35.079276 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260806 06:36:35.079983 11482 heartbeater.cc:347] Connected to a master server at 127.29.142.190:37129
I20260806 06:36:35.080067 11482 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:35.080276 11482 heartbeater.cc:511] Master 127.29.142.190:37129 requested a full tablet report, sending...
I20260806 06:36:35.080863 11210 ts_manager.cc:194] Registered new tserver with Master: fc04226f666a4fbe8814e0114a992bb4 (127.29.142.129:33727)
I20260806 06:36:35.081472 11210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.142.129:41459
W20260806 06:36:35.166802 11486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:35.167073 11486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:35.167115 11486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:35.168354 11486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:35.168411 11486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:35.168438 11486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:36:35.169885 11486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:35.170163 11486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:35.170531 11486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:35.176069 11499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:35.176074 11493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:35.176074 11494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:35.176357 11486 server_base.cc:1034] running on GCE node
I20260806 06:36:35.176618 11486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:35.176888 11486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:35.178072 11486 hybrid_clock.cc:648] HybridClock initialized: now 1785998195178055 us; error 34 us; skew 500 ppm
I20260806 06:36:35.179574 11486 webserver.cc:460] Webserver started at http://127.29.142.130:35659/ using document root <none> and password file <none>
I20260806 06:36:35.179872 11486 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:35.179949 11486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:35.180111 11486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:35.181257 11486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "a2b072f316984236a49e3d052812b715"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "0d6cbdbeb96455b35392039df8895f25"
server_key_iv: "0945255f71123133efa2e89dd72f4c2c"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.181654 11486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "a2b072f316984236a49e3d052812b715"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "0d6cbdbeb96455b35392039df8895f25"
server_key_iv: "0945255f71123133efa2e89dd72f4c2c"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.183135 11486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:36:35.183956 11505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.184149 11486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:35.184245 11486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "a2b072f316984236a49e3d052812b715"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "0d6cbdbeb96455b35392039df8895f25"
server_key_iv: "0945255f71123133efa2e89dd72f4c2c"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.184346 11486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:35.193387 11486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:35.193723 11486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:35.193887 11486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:35.194146 11486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:35.194497 11486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:35.194556 11486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.194604 11486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:35.194630 11486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.200846 11486 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:36053
I20260806 06:36:35.200907 11665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:36053 every 8 connection(s)
I20260806 06:36:35.201326 11486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:36:35.202394 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 11486
I20260806 06:36:35.202507 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:36:35.202759 30266 external_mini_cluster.cc:1428] Setting key 27469794934e7f9979b829b7d2a3750f
I20260806 06:36:35.204074 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37129
--builtin_ntp_servers=127.29.142.148:35323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 06:36:35.207901 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.202183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:56145 (local address 127.29.142.190:37129)
0806 06:36:35.202266 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:35.202269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:35.202572 (+ 303us) server_negotiation.cc:407] Connection header received
0806 06:36:35.203350 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:35.203351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:35.203361 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:35.203404 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:35.204411 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:35.204917 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:35.206229 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:35.206438 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:35.206534 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:35.207590 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:35.207591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:35.207592 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:35.207598 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:35.207617 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:35.207622 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:35.207691 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:35.207747 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 06:36:35.207778 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:36:35.208051 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.202153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37129 (local address 127.29.142.130:56145)
0806 06:36:35.202411 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:35.202464 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:35.203238 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:35.203486 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:35.203491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:35.203506 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:35.204300 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:35.204309 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:35.205016 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:35.205018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:35.206121 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:35.206124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:35.206546 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:35.206547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:35.206764 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:35.207434 (+ 670us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:35.207435 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:35.207467 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:35.207492 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:35.207692 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:35.207694 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:35.207697 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:36:35.207748 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:36:35.207832 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":105,"threads_started":1}
I20260806 06:36:35.208593 11669 heartbeater.cc:347] Connected to a master server at 127.29.142.190:37129
I20260806 06:36:35.208711 11669 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:35.208925 11669 heartbeater.cc:511] Master 127.29.142.190:37129 requested a full tablet report, sending...
I20260806 06:36:35.209367 11210 ts_manager.cc:194] Registered new tserver with Master: a2b072f316984236a49e3d052812b715 (127.29.142.130:36053)
I20260806 06:36:35.209765 11210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.142.130:56145
W20260806 06:36:35.291998 11672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:35.292236 11672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:35.292259 11672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:35.293458 11672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:35.293507 11672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:35.293533 11672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:36:35.295007 11672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:35.295288 11672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:35.295622 11672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:35.301029 11680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:35.300995 11684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:35.300995 11681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:35.301226 11672 server_base.cc:1034] running on GCE node
I20260806 06:36:35.301465 11672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:35.301739 11672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:35.302934 11672 hybrid_clock.cc:648] HybridClock initialized: now 1785998195302911 us; error 40 us; skew 500 ppm
I20260806 06:36:35.304322 11672 webserver.cc:460] Webserver started at http://127.29.142.131:35829/ using document root <none> and password file <none>
I20260806 06:36:35.304582 11672 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:35.304656 11672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:35.304833 11672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:35.305992 11672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "05e7d6ee6efd41d09827df2a53117e3a"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "7a649de9f6fe546f720a6e830a574ee6"
server_key_iv: "03a6b6a31bf6491c5c25524186898c18"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.306380 11672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "05e7d6ee6efd41d09827df2a53117e3a"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "7a649de9f6fe546f720a6e830a574ee6"
server_key_iv: "03a6b6a31bf6491c5c25524186898c18"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.307896 11672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:35.308709 11692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.308907 11672 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:35.309015 11672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "05e7d6ee6efd41d09827df2a53117e3a"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "7a649de9f6fe546f720a6e830a574ee6"
server_key_iv: "03a6b6a31bf6491c5c25524186898c18"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.309105 11672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:35.332221 11672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:35.332599 11672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:35.332784 11672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:35.333065 11672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:35.333420 11672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:35.333478 11672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.333516 11672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:35.333562 11672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.339675 11672 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:45441
I20260806 06:36:35.339722 11861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:45441 every 8 connection(s)
I20260806 06:36:35.340138 11672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:36:35.340718 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 11672
I20260806 06:36:35.340826 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:36:35.341045 30266 external_mini_cluster.cc:1428] Setting key 504eb7c3dcd47e45582044a9207d64cc
I20260806 06:36:35.345976 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.341022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:38099 (local address 127.29.142.190:37129)
0806 06:36:35.341097 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:35.341099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:35.341427 (+ 328us) server_negotiation.cc:407] Connection header received
0806 06:36:35.342186 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:35.342187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:35.342199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:35.342269 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:35.342962 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:35.343533 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:35.344618 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:35.344802 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:35.344860 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:35.345773 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:35.345773 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:35.345774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:35.345778 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:35.345790 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:35.345793 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:35.345831 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:35.345893 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 06:36:35.345917 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:36:35.346235 11864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.341024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37129 (local address 127.29.142.131:38099)
0806 06:36:35.341276 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:35.341328 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:35.342077 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:35.342330 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:35.342334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:35.342342 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:35.342891 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:35.342896 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:35.343648 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:35.343650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:35.344540 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:35.344543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:35.344880 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:35.344881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:35.344994 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:35.345638 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:35.345638 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:35.345671 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:35.345699 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:35.345833 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:35.345835 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:35.345837 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:36:35.345884 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:36:35.345965 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":109,"threads_started":1}
I20260806 06:36:35.346797 11862 heartbeater.cc:347] Connected to a master server at 127.29.142.190:37129
I20260806 06:36:35.346899 11862 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:35.347062 11862 heartbeater.cc:511] Master 127.29.142.190:37129 requested a full tablet report, sending...
I20260806 06:36:35.347442 11210 ts_manager.cc:194] Registered new tserver with Master: 05e7d6ee6efd41d09827df2a53117e3a (127.29.142.131:45441)
I20260806 06:36:35.347800 11210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.142.131:38099
I20260806 06:36:35.353133 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:36:35.355280 11264 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:35.354746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60780 (local address 127.29.142.190:37129)
0806 06:36:35.354825 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:35.354827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:35.354940 (+ 113us) server_negotiation.cc:407] Connection header received
0806 06:36:35.354979 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:35.354980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:35.354993 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:35.355039 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:35.355216 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60780: BlockingRecv error: recv got EOF from 127.0.0.1:60780 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:36:35.355414 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 11290
I20260806 06:36:35.362732 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 11486
I20260806 06:36:35.378777 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 11672
I20260806 06:36:35.391244 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 11169
2026-08-06T06:36:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (685 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T06:36:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:35Z Disabled control of system clock
I20260806 06:36:35.424157 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:34687
--webserver_interface=127.29.142.190
--webserver_port=0
--builtin_ntp_servers=127.29.142.148:42291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:34687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:36:35.508628 11877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:35.508867 11877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:35.508891 11877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:35.509991 11877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:35.510033 11877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:35.510048 11877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:35.510061 11877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:35.510073 11877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:35.511579 11877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:42291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:34687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:34687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:35.511842 11877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:35.512111 11877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:35.517534 11888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:35.517696 11877 server_base.cc:1034] running on GCE node
W20260806 06:36:35.517536 11883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:35.517536 11884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:35.518106 11877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:35.518436 11877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:35.519586 11877 hybrid_clock.cc:648] HybridClock initialized: now 1785998195519561 us; error 44 us; skew 500 ppm
I20260806 06:36:35.521061 11877 webserver.cc:460] Webserver started at http://127.29.142.190:32939/ using document root <none> and password file <none>
I20260806 06:36:35.521345 11877 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:35.521395 11877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:35.521553 11877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:35.522782 11877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "8fb336c9dfa8487cbb6742fc2a5fcae0"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "a78eb9033c9a49d7a26be70797a9b211"
server_key_iv: "5b03a801ec27fb0ebfa0e1291d990acc"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.523172 11877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "8fb336c9dfa8487cbb6742fc2a5fcae0"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "a78eb9033c9a49d7a26be70797a9b211"
server_key_iv: "5b03a801ec27fb0ebfa0e1291d990acc"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.524500 11877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:36:35.525241 11898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.525421 11877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:36:35.525518 11877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "8fb336c9dfa8487cbb6742fc2a5fcae0"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "a78eb9033c9a49d7a26be70797a9b211"
server_key_iv: "5b03a801ec27fb0ebfa0e1291d990acc"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.525599 11877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:35.529771 11877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:35.530105 11877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:35.530289 11877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:35.535072 11877 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:34687
I20260806 06:36:35.535092 11973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:34687 every 8 connection(s)
I20260806 06:36:35.535534 11877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:35.536504 11974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:35.539222 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 11877
I20260806 06:36:35.539323 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:35.539531 30266 external_mini_cluster.cc:1428] Setting key 8da4932916b063fd8841cd2dbd83983b
I20260806 06:36:35.541711 11974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0: Bootstrap starting.
I20260806 06:36:35.542443 11974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:35.542865 11974 log.cc:824] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:35.547452 11974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0: No bootstrap required, opened a new log
I20260806 06:36:35.549001 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.542792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37524 (local address 127.29.142.190:34687)
0806 06:36:35.543098 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:35.543108 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:35.546875 (+ 3767us) server_negotiation.cc:407] Connection header received
0806 06:36:35.547469 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:35.547478 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:35.547523 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:35.547733 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:35.548495 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:35.548498 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:35.548499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:35.548502 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:35.548543 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:35.548555 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:35.548632 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:35.548708 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:36:35.548787 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260806 06:36:35.549530 11974 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb336c9dfa8487cbb6742fc2a5fcae0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 34687 } }
I20260806 06:36:35.549713 11974 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:35.549755 11974 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb336c9dfa8487cbb6742fc2a5fcae0, State: Initialized, Role: FOLLOWER
I20260806 06:36:35.549912 11974 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb336c9dfa8487cbb6742fc2a5fcae0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 34687 } }
I20260806 06:36:35.549996 11974 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:35.550019 11974 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:35.550048 11974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:35.551138 11974 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb336c9dfa8487cbb6742fc2a5fcae0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 34687 } }
I20260806 06:36:35.551267 11974 leader_election.cc:302] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fb336c9dfa8487cbb6742fc2a5fcae0; no voters:
I20260806 06:36:35.551512 11974 leader_election.cc:288] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:35.551592 11979 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:35.551751 11979 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [term 1 LEADER]: Becoming Leader. State: Replica: 8fb336c9dfa8487cbb6742fc2a5fcae0, State: Running, Role: LEADER
I20260806 06:36:35.551844 11974 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:35.551858 11979 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb336c9dfa8487cbb6742fc2a5fcae0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 34687 } }
I20260806 06:36:35.552233 11981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fb336c9dfa8487cbb6742fc2a5fcae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb336c9dfa8487cbb6742fc2a5fcae0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 34687 } } }
I20260806 06:36:35.552361 11981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:35.552232 11980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fb336c9dfa8487cbb6742fc2a5fcae0. Latest consensus state: current_term: 1 leader_uuid: "8fb336c9dfa8487cbb6742fc2a5fcae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb336c9dfa8487cbb6742fc2a5fcae0" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 34687 } } }
I20260806 06:36:35.552516 11980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0 [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:35.552608 11986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:35.553118 11986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:35.554594 11986 catalog_manager.cc:1347] Generated new cluster ID: b7e2e971e21047a899754f1a45eba486
I20260806 06:36:35.554672 11986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:36:35.563779 11986 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:35.563985 11986 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:35.571944 11986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fb336c9dfa8487cbb6742fc2a5fcae0: Generated new TSK 0
I20260806 06:36:35.572175 11986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:36:35.582504 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:34687
--builtin_ntp_servers=127.29.142.148:42291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:36:35.670691 12006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:35.670960 12006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:35.670986 12006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:35.672320 12006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:35.672369 12006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:35.672397 12006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:36:35.673844 12006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:42291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:34687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:35.674121 12006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:35.674456 12006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:35.680371 12015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:35.680389 12020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:35.680549 12006 server_base.cc:1034] running on GCE node
W20260806 06:36:35.680394 12016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:35.680848 12006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:35.681123 12006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:35.682330 12006 hybrid_clock.cc:648] HybridClock initialized: now 1785998195682320 us; error 33 us; skew 500 ppm
I20260806 06:36:35.688654 12006 webserver.cc:460] Webserver started at http://127.29.142.129:42273/ using document root <none> and password file <none>
I20260806 06:36:35.688990 12006 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:35.689073 12006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:35.689232 12006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:35.690380 12006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "86ab06d4ed904166a7e20a530a6bb621"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "300a8883d3a15cc253af01626a67c201"
server_key_iv: "6842580f3c12e4967c259645bbce7613"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.690815 12006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "86ab06d4ed904166a7e20a530a6bb621"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "300a8883d3a15cc253af01626a67c201"
server_key_iv: "6842580f3c12e4967c259645bbce7613"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.692250 12006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:35.693074 12028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.693254 12006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:35.693358 12006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "86ab06d4ed904166a7e20a530a6bb621"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "300a8883d3a15cc253af01626a67c201"
server_key_iv: "6842580f3c12e4967c259645bbce7613"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.693442 12006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:35.697077 12006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:35.697361 12006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:35.697505 12006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:35.697721 12006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:35.698101 12006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:35.698160 12006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.698201 12006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:35.698230 12006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.704417 12006 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:40993
I20260806 06:36:35.704458 12183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:40993 every 8 connection(s)
I20260806 06:36:35.704846 12006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:36:35.708151 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.705656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:53357 (local address 127.29.142.190:34687)
0806 06:36:35.705749 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:35.705764 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:35.705988 (+ 224us) server_negotiation.cc:407] Connection header received
0806 06:36:35.706550 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:35.706551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:35.706560 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:35.706613 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:35.707863 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:35.707864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:35.707865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:35.707870 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:35.707886 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:35.707891 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:35.707964 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:35.708058 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 06:36:35.708075 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:36:35.708396 12186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.705654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:34687 (local address 127.29.142.129:53357)
0806 06:36:35.705856 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:35.705894 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:35.706422 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:35.706716 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:35.706720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:35.706731 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:35.707701 (+ 970us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:35.707701 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:35.707733 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:35.707765 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:35.707962 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:35.707968 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:35.707972 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:36:35.708070 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 06:36:35.708161 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260806 06:36:35.708407 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 12006
I20260806 06:36:35.708523 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:36:35.708721 30266 external_mini_cluster.cc:1428] Setting key 1a20a2a9f98b76e879852b48404de82b
I20260806 06:36:35.709227 12184 heartbeater.cc:347] Connected to a master server at 127.29.142.190:34687
I20260806 06:36:35.709338 12184 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:35.709510 12184 heartbeater.cc:511] Master 127.29.142.190:34687 requested a full tablet report, sending...
I20260806 06:36:35.709878 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:34687
--builtin_ntp_servers=127.29.142.148:42291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 06:36:35.710150 11925 ts_manager.cc:194] Registered new tserver with Master: 86ab06d4ed904166a7e20a530a6bb621 (127.29.142.129:40993)
W20260806 06:36:35.798333 12188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:35.798678 12188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:35.798718 12188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:35.800801 12188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:35.800873 12188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:35.800922 12188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:36:35.803440 12188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:42291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:34687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:35.803761 12188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:35.804167 12188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:35.814141 12197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:35.814311 12188 server_base.cc:1034] running on GCE node
W20260806 06:36:35.814141 12196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:35.814153 12199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:35.814672 12188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:35.815009 12188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:35.816144 12188 hybrid_clock.cc:648] HybridClock initialized: now 1785998195816125 us; error 39 us; skew 500 ppm
I20260806 06:36:35.817987 12188 webserver.cc:460] Webserver started at http://127.29.142.130:46443/ using document root <none> and password file <none>
I20260806 06:36:35.818317 12188 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:35.818387 12188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:35.818542 12188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:35.819751 12188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "a158ca4807df4c0f881af5d8288f6e22"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "d9344192203e5648776a26d95ad0f9fd"
server_key_iv: "d2dc1daead45e386bb9944e1ba382de5"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.820089 12188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "a158ca4807df4c0f881af5d8288f6e22"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "d9344192203e5648776a26d95ad0f9fd"
server_key_iv: "d2dc1daead45e386bb9944e1ba382de5"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.821583 12188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:36:35.822568 12209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.822793 12188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:35.822886 12188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "a158ca4807df4c0f881af5d8288f6e22"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "d9344192203e5648776a26d95ad0f9fd"
server_key_iv: "d2dc1daead45e386bb9944e1ba382de5"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.822988 12188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:35.827458 12188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:35.827834 12188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:35.827987 12188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:35.828281 12188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:35.828697 12188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:35.828780 12188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.828815 12188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:35.828835 12188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.837577 12188 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:39321
I20260806 06:36:35.837661 12360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:39321 every 8 connection(s)
I20260806 06:36:35.838133 12188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:36:35.842422 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.839057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:42073 (local address 127.29.142.190:34687)
0806 06:36:35.839110 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:35.839113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:35.839476 (+ 363us) server_negotiation.cc:407] Connection header received
0806 06:36:35.840308 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:35.840310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:35.840317 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:35.840383 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:35.842120 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:35.842121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:35.842122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:35.842127 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:35.842147 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:35.842151 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:35.842216 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:35.842322 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 06:36:35.842338 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:36:35.842728 12363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.839059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:34687 (local address 127.29.142.130:42073)
0806 06:36:35.839316 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:35.839370 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:35.840170 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:35.840486 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:35.840491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:35.840502 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:35.841929 (+ 1427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:35.841929 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:35.841974 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:35.842017 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:35.842238 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:35.842243 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:35.842247 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:36:35.842336 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 06:36:35.842417 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":110,"threads_started":1}
I20260806 06:36:35.843212 12361 heartbeater.cc:347] Connected to a master server at 127.29.142.190:34687
I20260806 06:36:35.843333 12361 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:35.843494 12361 heartbeater.cc:511] Master 127.29.142.190:34687 requested a full tablet report, sending...
I20260806 06:36:35.843905 11925 ts_manager.cc:194] Registered new tserver with Master: a158ca4807df4c0f881af5d8288f6e22 (127.29.142.130:39321)
I20260806 06:36:35.847261 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 12188
I20260806 06:36:35.847376 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:36:35.847579 30266 external_mini_cluster.cc:1428] Setting key f31e6bb80a147c625d400cf370fad3d7
I20260806 06:36:35.848594 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:34687
--builtin_ntp_servers=127.29.142.148:42291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:36:35.936905 12364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:35.937152 12364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:35.937177 12364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:35.938354 12364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:35.938395 12364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:35.938422 12364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:36:35.939767 12364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:42291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:34687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:35.939978 12364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:35.940245 12364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:35.945653 12378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:35.945680 12374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:35.945664 12376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:35.946022 12364 server_base.cc:1034] running on GCE node
I20260806 06:36:35.946277 12364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:35.946590 12364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:35.947818 12364 hybrid_clock.cc:648] HybridClock initialized: now 1785998195947781 us; error 43 us; skew 500 ppm
I20260806 06:36:35.949397 12364 webserver.cc:460] Webserver started at http://127.29.142.131:42263/ using document root <none> and password file <none>
I20260806 06:36:35.949671 12364 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:35.949745 12364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:35.949913 12364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:35.951208 12364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "d13a60871c274b0fb0994113f48bed78"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "13e13e3657e42ca0ebceba535dfab73d"
server_key_iv: "11cc7e65c18e5a074460223fea1e6d04"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.951671 12364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "d13a60871c274b0fb0994113f48bed78"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "13e13e3657e42ca0ebceba535dfab73d"
server_key_iv: "11cc7e65c18e5a074460223fea1e6d04"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.953285 12364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:35.954120 12388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.954334 12364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:36:35.954444 12364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "d13a60871c274b0fb0994113f48bed78"
format_stamp: "Formatted at 2026-08-06 06:36:35 on dist-test-slave-3kk6"
server_key: "13e13e3657e42ca0ebceba535dfab73d"
server_key_iv: "11cc7e65c18e5a074460223fea1e6d04"
server_key_version: "encryptionkey@0"
I20260806 06:36:35.954524 12364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:35.958105 12364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:35.958403 12364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:35.958557 12364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:35.958825 12364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:35.959177 12364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:35.959230 12364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.959272 12364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:35.959301 12364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:35.965593 12364 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:46543
I20260806 06:36:35.965651 12535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:46543 every 8 connection(s)
I20260806 06:36:35.966027 12364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:36:35.969448 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.966863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:44913 (local address 127.29.142.190:34687)
0806 06:36:35.966914 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:35.966916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:35.967244 (+ 328us) server_negotiation.cc:407] Connection header received
0806 06:36:35.967799 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:35.967800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:35.967809 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:35.967862 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:35.969142 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:35.969144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:35.969145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:36:35.969151 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:35.969176 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:36:35.969181 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:35.969267 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:35.969352 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 06:36:35.969369 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:36:35.969651 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:35.966878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:34687 (local address 127.29.142.131:44913)
0806 06:36:35.967057 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:35.967123 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:35.967670 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:35.967936 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:35.967941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:35.967951 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:35.968971 (+ 1020us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:36:35.968971 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:35.969001 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:36:35.969031 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:35.969267 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:35.969275 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:35.969278 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:36:35.969363 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:36:35.969431 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260806 06:36:35.970124 12538 heartbeater.cc:347] Connected to a master server at 127.29.142.190:34687
I20260806 06:36:35.970208 12538 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:35.970391 12538 heartbeater.cc:511] Master 127.29.142.190:34687 requested a full tablet report, sending...
I20260806 06:36:35.970836 11925 ts_manager.cc:194] Registered new tserver with Master: d13a60871c274b0fb0994113f48bed78 (127.29.142.131:46543)
I20260806 06:36:35.975273 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 12364
I20260806 06:36:35.975371 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:36:35.975526 30266 external_mini_cluster.cc:1428] Setting key 39cb141c7dce068ac1e4907977d09d17
I20260806 06:36:35.976553 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:36:35.978453 11977 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:36:35.977967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37530 (local address 127.29.142.190:34687)
0806 06:36:35.978022 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:35.978025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:35.978145 (+ 120us) server_negotiation.cc:407] Connection header received
0806 06:36:35.978184 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:35.978185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:35.978191 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:35.978243 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:35.978383 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37530: BlockingRecv error: recv got EOF from 127.0.0.1:37530 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:36:35.978503 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 12006
I20260806 06:36:35.992892 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 12188
I20260806 06:36:36.008287 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 12364
I20260806 06:36:36.015909 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 11877
2026-08-06T06:36:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (624 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:36:36 dist-test-slave-3kk6 krb5kdc[12554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:36:36 dist-test-slave-3kk6 krb5kdc[12554](info): set up 2 sockets
Aug 06 06:36:36 dist-test-slave-3kk6 krb5kdc[12554](info): commencing operation
krb5kdc: starting...
W20260806 06:36:37.257795 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:36:37 dist-test-slave-3kk6 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:36:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:36:37.385845 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:39475
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:39475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:37.471906 12579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:37.472189 12579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:37.472234 12579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:37.473452 12579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:36:37.473528 12579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:37.473557 12579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:37.473590 12579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:36:37.473615 12579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:36:37.475260 12579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:39475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:39475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:37.475554 12579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:37.475812 12579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:37.481143 12587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:37.481153 12592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:37.481191 12579 server_base.cc:1034] running on GCE node
W20260806 06:36:37.481168 12589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:37.481626 12579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:37.481935 12579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:37.483115 12579 hybrid_clock.cc:648] HybridClock initialized: now 1785998197483092 us; error 38 us; skew 500 ppm
Aug 06 06:36:37 dist-test-slave-3kk6 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998197, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:37.485781 12579 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:36:37.486305 12579 webserver.cc:460] Webserver started at http://127.29.142.190:34095/ using document root <none> and password file <none>
I20260806 06:36:37.486529 12579 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:37.486572 12579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:37.486732 12579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:37.487829 12579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "04d044207f5744258617c6f57e40074b"
format_stamp: "Formatted at 2026-08-06 06:36:37 on dist-test-slave-3kk6"
server_key: "3da785ba0dc2a8a139f12f2bbd0d3026"
server_key_iv: "26aeb78e43c577f85cfeae2f24c37fe3"
server_key_version: "encryptionkey@0"
I20260806 06:36:37.488162 12579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "04d044207f5744258617c6f57e40074b"
format_stamp: "Formatted at 2026-08-06 06:36:37 on dist-test-slave-3kk6"
server_key: "3da785ba0dc2a8a139f12f2bbd0d3026"
server_key_iv: "26aeb78e43c577f85cfeae2f24c37fe3"
server_key_version: "encryptionkey@0"
I20260806 06:36:37.489564 12579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:36:37.490334 12602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:37.490536 12579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:37.490607 12579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "04d044207f5744258617c6f57e40074b"
format_stamp: "Formatted at 2026-08-06 06:36:37 on dist-test-slave-3kk6"
server_key: "3da785ba0dc2a8a139f12f2bbd0d3026"
server_key_iv: "26aeb78e43c577f85cfeae2f24c37fe3"
server_key_version: "encryptionkey@0"
I20260806 06:36:37.490728 12579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:37.512637 12579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:37.515110 12579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:37.515262 12579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:37.519654 12579 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:39475
I20260806 06:36:37.519692 12670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:39475 every 8 connection(s)
I20260806 06:36:37.520112 12579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:36:37.521252 12671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:37.522058 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 12579
I20260806 06:36:37.522217 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:36:37.522415 30266 external_mini_cluster.cc:1428] Setting key 178daf9027e8828b13db050197271a0c
I20260806 06:36:37.524876 12671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b: Bootstrap starting.
I20260806 06:36:37.525674 12671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:37.526008 12671 log.cc:824] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:37.526984 12671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b: No bootstrap required, opened a new log
I20260806 06:36:37.529546 12671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d044207f5744258617c6f57e40074b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39475 } }
I20260806 06:36:37.529739 12671 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:37.529795 12671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d044207f5744258617c6f57e40074b, State: Initialized, Role: FOLLOWER
I20260806 06:36:37.529945 12671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d044207f5744258617c6f57e40074b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39475 } }
I20260806 06:36:37.530076 12671 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:36:37.530126 12671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:36:37.530180 12671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [term 0 FOLLOWER]: Advancing to term 1
Aug 06 06:36:37 dist-test-slave-3kk6 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:37.531032 12671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d044207f5744258617c6f57e40074b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39475 } }
I20260806 06:36:37.531179 12671 leader_election.cc:302] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04d044207f5744258617c6f57e40074b; no voters:
I20260806 06:36:37.531394 12671 leader_election.cc:288] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:36:37.531477 12678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:37.531657 12678 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [term 1 LEADER]: Becoming Leader. State: Replica: 04d044207f5744258617c6f57e40074b, State: Running, Role: LEADER
I20260806 06:36:37.531749 12671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:36:37.531773 12678 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d044207f5744258617c6f57e40074b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39475 } }
I20260806 06:36:37.532124 12678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04d044207f5744258617c6f57e40074b. Latest consensus state: current_term: 1 leader_uuid: "04d044207f5744258617c6f57e40074b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d044207f5744258617c6f57e40074b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39475 } } }
I20260806 06:36:37.532212 12678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:37.532298 12681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04d044207f5744258617c6f57e40074b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d044207f5744258617c6f57e40074b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39475 } } }
I20260806 06:36:37.532450 12681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b [sys.catalog]: This master's current role is: LEADER
I20260806 06:36:37.533067 12685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:36:37.533603 12685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:36:37.535074 12676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:37.523092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60680 (local address 127.29.142.190:39475)
0806 06:36:37.523313 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:37.523324 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:37.523363 (+ 39us) server_negotiation.cc:407] Connection header received
0806 06:36:37.523974 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:37.523984 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:37.524054 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:37.524219 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:37.524865 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:37.526305 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:37.527338 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:37.527969 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:37.528101 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:37.530826 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:37.530831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:37.530831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:37.530843 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:37.533135 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:37.533693 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:37.533697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:37.533698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:37.533822 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:37.534078 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:37.534079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:37.534079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:37.534341 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:37.534516 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:37.534633 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 06:36:37.534806 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":95,"threads_started":1}
I20260806 06:36:37.535986 12685 catalog_manager.cc:1347] Generated new cluster ID: 7b63eeee3f99420aa7e5ca1b312b43e3
I20260806 06:36:37.536031 12685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:36:37.545271 12685 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:36:37.546067 12685 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:36:37.555385 12685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b: Generated new TSK 0
I20260806 06:36:37.555644 12685 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:36:37.589263 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39475
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:37.676610 12703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:37.676867 12703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:37.676894 12703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:37.678143 12703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:37.678194 12703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:37.678223 12703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:36:37.679808 12703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:37.680097 12703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:37.680356 12703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:37.685981 12712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:37.686026 12711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:37.686208 12703 server_base.cc:1034] running on GCE node
W20260806 06:36:37.685981 12714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:37.686503 12703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:37.686820 12703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:37.687975 12703 hybrid_clock.cc:648] HybridClock initialized: now 1785998197687962 us; error 30 us; skew 500 ppm
Aug 06 06:36:37 dist-test-slave-3kk6 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998197, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:37.690981 12703 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:36:37.691476 12703 webserver.cc:460] Webserver started at http://127.29.142.129:44663/ using document root <none> and password file <none>
I20260806 06:36:37.691743 12703 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:37.691819 12703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:37.691978 12703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:37.693117 12703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "26645061ced64ece9bb333be48756930"
format_stamp: "Formatted at 2026-08-06 06:36:37 on dist-test-slave-3kk6"
server_key: "a0d3eedd28d1d5a11cd08590c7a40997"
server_key_iv: "cd22a88f33c3a256ae935e226f1d52c6"
server_key_version: "encryptionkey@0"
I20260806 06:36:37.693581 12703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "26645061ced64ece9bb333be48756930"
format_stamp: "Formatted at 2026-08-06 06:36:37 on dist-test-slave-3kk6"
server_key: "a0d3eedd28d1d5a11cd08590c7a40997"
server_key_iv: "cd22a88f33c3a256ae935e226f1d52c6"
server_key_version: "encryptionkey@0"
I20260806 06:36:37.695152 12703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:36:37.695981 12725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:37.696182 12703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:36:37.696295 12703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "26645061ced64ece9bb333be48756930"
format_stamp: "Formatted at 2026-08-06 06:36:37 on dist-test-slave-3kk6"
server_key: "a0d3eedd28d1d5a11cd08590c7a40997"
server_key_iv: "cd22a88f33c3a256ae935e226f1d52c6"
server_key_version: "encryptionkey@0"
I20260806 06:36:37.696385 12703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:37.706426 12703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:37.709168 12703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:37.709378 12703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:37.709658 12703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:37.710023 12703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:37.710083 12703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:37.710126 12703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:37.710156 12703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:37.716447 12703 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:38683
I20260806 06:36:37.716521 12881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:38683 every 8 connection(s)
I20260806 06:36:37.716898 12703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:36:37 dist-test-slave-3kk6 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998197, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:37.725375 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 12703
I20260806 06:36:37.725502 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:36:37.725739 30266 external_mini_cluster.cc:1428] Setting key 8af9c4f702fbff8b36faafbaed8e23bd
I20260806 06:36:37.727311 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:37.717676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39475 (local address 127.29.142.129:47993)
0806 06:36:37.717870 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:37.717915 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:37.718439 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:37.718714 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:37.718719 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:37.719351 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:37.719560 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:37.719565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:37.720297 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:37.720298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:37.721136 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:37.721137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:37.721545 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:37.721546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:37.721651 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:37.722221 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:37.722230 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:37.723749 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:37.725521 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:37.725529 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:37.725532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:37.725857 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:37.726120 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:37.726121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:37.726121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:37.726294 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:37.726748 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:37.726759 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:37.726887 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 06:36:37.726982 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 06:36:37.727118 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":115,"threads_started":1}
I20260806 06:36:37.727346 12676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:37.717741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:47993 (local address 127.29.142.190:39475)
0806 06:36:37.717791 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:37.717793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:37.718024 (+ 231us) server_negotiation.cc:407] Connection header received
0806 06:36:37.718528 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:37.718529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:37.718539 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:37.718605 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:37.719674 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:37.720189 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:37.721246 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:37.721461 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:37.721523 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:37.723854 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:37.723856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:37.723856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:37.723865 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:37.725392 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:37.725957 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:37.725958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:37.725959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:37.726032 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:37.726421 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:37.726423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:37.726424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:37.726622 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:37.726756 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:37.727039 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 06:36:37.727182 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:36:37.728911 12883 heartbeater.cc:347] Connected to a master server at 127.29.142.190:39475
I20260806 06:36:37.729058 12883 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:37.729281 12883 heartbeater.cc:511] Master 127.29.142.190:39475 requested a full tablet report, sending...
I20260806 06:36:37.730039 12622 ts_manager.cc:194] Registered new tserver with Master: 26645061ced64ece9bb333be48756930 (127.29.142.129:38683)
I20260806 06:36:37.731191 12622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:47993
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:36:37.758483 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39475
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:37.844765 12890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:37.844991 12890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:37.845016 12890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:37.846107 12890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:37.846150 12890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:37.846176 12890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:36:37.847600 12890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:37.847793 12890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:37.847975 12890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:36:37.853153 12899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:37.853153 12895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:37.853245 12896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:37.853514 12890 server_base.cc:1034] running on GCE node
I20260806 06:36:37.853797 12890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:37.854017 12890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:37.855187 12890 hybrid_clock.cc:648] HybridClock initialized: now 1785998197855160 us; error 39 us; skew 500 ppm
Aug 06 06:36:37 dist-test-slave-3kk6 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998197, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:37.857899 12890 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:36:37.858374 12890 webserver.cc:460] Webserver started at http://127.29.142.130:41955/ using document root <none> and password file <none>
I20260806 06:36:37.858613 12890 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:37.858716 12890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:37.858875 12890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:37.859983 12890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "3f1a7ab672db4aa6bc6d1f892f33965e"
format_stamp: "Formatted at 2026-08-06 06:36:37 on dist-test-slave-3kk6"
server_key: "8248bcee6ed15a57ffa3aa595046a449"
server_key_iv: "08d6cc811bcd74b30bbcf4ce49341a46"
server_key_version: "encryptionkey@0"
I20260806 06:36:37.860371 12890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "3f1a7ab672db4aa6bc6d1f892f33965e"
format_stamp: "Formatted at 2026-08-06 06:36:37 on dist-test-slave-3kk6"
server_key: "8248bcee6ed15a57ffa3aa595046a449"
server_key_iv: "08d6cc811bcd74b30bbcf4ce49341a46"
server_key_version: "encryptionkey@0"
I20260806 06:36:37.861792 12890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:36:37.862547 12909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:37.862843 12890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:36:37.862953 12890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "3f1a7ab672db4aa6bc6d1f892f33965e"
format_stamp: "Formatted at 2026-08-06 06:36:37 on dist-test-slave-3kk6"
server_key: "8248bcee6ed15a57ffa3aa595046a449"
server_key_iv: "08d6cc811bcd74b30bbcf4ce49341a46"
server_key_version: "encryptionkey@0"
I20260806 06:36:37.863032 12890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:37.871675 12890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:37.873613 12890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:37.873798 12890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:37.874065 12890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:37.874409 12890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:37.874465 12890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:37.874507 12890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:37.874550 12890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:37.880546 12890 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:35087
I20260806 06:36:37.880617 13060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:35087 every 8 connection(s)
I20260806 06:36:37.880990 12890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:36:37.884824 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 12890
I20260806 06:36:37.884958 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:36:37.885174 30266 external_mini_cluster.cc:1428] Setting key a86296c444fb707dd58980737a6c8e63
Aug 06 06:36:37 dist-test-slave-3kk6 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998197, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:37.891618 12676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:37.881852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:45483 (local address 127.29.142.190:39475)
0806 06:36:37.881905 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:37.881907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:37.882138 (+ 231us) server_negotiation.cc:407] Connection header received
0806 06:36:37.882781 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:37.882783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:37.882799 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:37.882865 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:37.883915 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:37.884471 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:37.885730 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:37.886000 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:37.886080 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:37.888475 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:37.888476 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:37.888477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:37.888485 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:37.889978 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:37.890511 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:37.890512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:37.890513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:37.890570 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:37.890902 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:37.890903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:37.890903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:37.891019 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:37.891075 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:37.891471 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 06:36:37.891549 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:36:37.891943 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:37.881794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39475 (local address 127.29.142.130:45483)
0806 06:36:37.882003 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:37.882042 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:37.882600 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:37.882952 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:37.882957 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:37.883578 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:37.883786 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:37.883791 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:37.884575 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:37.884576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:37.885641 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:37.885643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:37.886086 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:37.886087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:37.886232 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:37.886896 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:37.886905 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:37.888365 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:37.890063 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:37.890071 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:37.890074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:37.890424 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:37.890640 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:37.890641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:37.890642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:37.890829 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:37.891092 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:37.891100 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:37.891298 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 06:36:37.891423 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 06:36:37.891626 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":120,"threads_started":1}
I20260806 06:36:37.892498 13061 heartbeater.cc:347] Connected to a master server at 127.29.142.190:39475
I20260806 06:36:37.892612 13061 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:37.892830 13061 heartbeater.cc:511] Master 127.29.142.190:39475 requested a full tablet report, sending...
I20260806 06:36:37.893266 12622 ts_manager.cc:194] Registered new tserver with Master: 3f1a7ab672db4aa6bc6d1f892f33965e (127.29.142.130:35087)
I20260806 06:36:37.893653 12622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:45483
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:36:37.914477 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39475
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:36:38.000295 13067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:36:38.000532 13067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:36:38.000560 13067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:36:38.001793 13067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:36:38.001849 13067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:36:38.001878 13067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:36:38.003463 13067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:39475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:36:38.003738 13067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:36:38.004000 13067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:36:38.009699 13067 server_base.cc:1034] running on GCE node
W20260806 06:36:38.009601 13076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:38.009604 13073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:36:38.009604 13074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:36:38.010142 13067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:36:38.010381 13067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:36:38.011531 13067 hybrid_clock.cc:648] HybridClock initialized: now 1785998198011497 us; error 57 us; skew 500 ppm
Aug 06 06:36:38 dist-test-slave-3kk6 krb5kdc[12554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998198, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:36:38.014334 13067 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:36:38.014927 13067 webserver.cc:460] Webserver started at http://127.29.142.131:38505/ using document root <none> and password file <none>
I20260806 06:36:38.015168 13067 fs_manager.cc:359] Metadata directory not provided
I20260806 06:36:38.015218 13067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:36:38.015359 13067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:36:38.016563 13067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "3244c385d5be44aebca188dddc19f6cb"
format_stamp: "Formatted at 2026-08-06 06:36:38 on dist-test-slave-3kk6"
server_key: "dcbbafcaea04deab23bdfcbe1762799e"
server_key_iv: "609ab9181cb0a7738ede2fc7c4cfd73a"
server_key_version: "encryptionkey@0"
I20260806 06:36:38.016952 13067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "3244c385d5be44aebca188dddc19f6cb"
format_stamp: "Formatted at 2026-08-06 06:36:38 on dist-test-slave-3kk6"
server_key: "dcbbafcaea04deab23bdfcbe1762799e"
server_key_iv: "609ab9181cb0a7738ede2fc7c4cfd73a"
server_key_version: "encryptionkey@0"
I20260806 06:36:38.018370 13067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:36:38.019212 13083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:38.019397 13067 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:38.019512 13067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "3244c385d5be44aebca188dddc19f6cb"
format_stamp: "Formatted at 2026-08-06 06:36:38 on dist-test-slave-3kk6"
server_key: "dcbbafcaea04deab23bdfcbe1762799e"
server_key_iv: "609ab9181cb0a7738ede2fc7c4cfd73a"
server_key_version: "encryptionkey@0"
I20260806 06:36:38.019593 13067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:36:38.041560 13067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:36:38.044291 13067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:36:38.044502 13067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:36:38.044793 13067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:36:38.045135 13067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:36:38.045192 13067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:38.045240 13067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:36:38.045264 13067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:36:38.051824 13067 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:37881
I20260806 06:36:38.051867 13237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:37881 every 8 connection(s)
I20260806 06:36:38.052253 13067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:36:38 dist-test-slave-3kk6 krb5kdc[12554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998198, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:36:38.060585 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 13067
I20260806 06:36:38.060707 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:36:38.060981 30266 external_mini_cluster.cc:1428] Setting key f69185e0c02ef4810997d6943d4853b4
I20260806 06:36:38.063230 12676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.053095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:40049 (local address 127.29.142.190:39475)
0806 06:36:38.053142 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:38.053144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:38.053368 (+ 224us) server_negotiation.cc:407] Connection header received
0806 06:36:38.053952 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:38.053953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:38.053966 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:38.054042 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:38.055218 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.055883 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.057019 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.057223 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.057288 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.059719 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:38.059721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:38.059721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:38.059729 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:38.061589 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:38.062208 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:38.062209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:38.062209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:38.062278 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:38.062519 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:38.062520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:38.062520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:38.062635 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:38.062749 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:38.063069 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 06:36:38.063147 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:36:38.063467 13240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.053038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39475 (local address 127.29.142.131:40049)
0806 06:36:38.053231 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:38.053279 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:38.053823 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:38.054172 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:38.054176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:38.054851 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:36:38.055078 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:38.055084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.056011 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.056012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:38.056915 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:38.056918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.057311 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.057312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:38.057424 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.057992 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:36:38.058001 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:36:38.059591 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:36:38.061753 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:38.061764 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:38.061767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:38.062134 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:38.062341 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:36:38.062341 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:36:38.062341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:36:38.062461 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:36:38.062780 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:36:38.062789 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:36:38.062939 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 06:36:38.063038 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 06:36:38.063181 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
I20260806 06:36:38.064040 13238 heartbeater.cc:347] Connected to a master server at 127.29.142.190:39475
I20260806 06:36:38.064157 13238 heartbeater.cc:464] Registering TS with master...
I20260806 06:36:38.064361 13238 heartbeater.cc:511] Master 127.29.142.190:39475 requested a full tablet report, sending...
I20260806 06:36:38.064781 12622 ts_manager.cc:194] Registered new tserver with Master: 3244c385d5be44aebca188dddc19f6cb (127.29.142.131:37881)
I20260806 06:36:38.065169 12622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:40049
I20260806 06:36:38.073057 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:36:38.080868 12676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.074501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60692 (local address 127.29.142.190:39475)
0806 06:36:38.074575 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:38.074577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:38.074733 (+ 156us) server_negotiation.cc:407] Connection header received
0806 06:36:38.074819 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:38.074820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:38.074830 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:38.074888 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:36:38.075457 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.075946 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.076662 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.076850 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.076905 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.078161 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:36:38.078162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:36:38.078162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:36:38.078169 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:36:38.079726 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:38.080060 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:38.080061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:38.080062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:38.080116 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:36:38.080363 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:36:38.080364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:36:38.080365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:36:38.080540 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:36:38.080636 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:38.080729 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 06:36:38.080796 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:36:38.082868 12622 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:36:38.084619 12622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:36:38.091943 13258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.087558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54076 (local address 127.29.142.130:35087)
0806 06:36:38.088129 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:38.088133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:38.088144 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:38.088185 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:38.088187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:38.088202 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:38.088309 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:38.088451 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.089165 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.090766 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.091446 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.091519 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.091568 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:38.091794 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 06:36:38.091843 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":111,"threads_started":1}
I20260806 06:36:38.091951 13252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.087419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:35087 (local address 127.0.0.1:54076)
0806 06:36:38.087766 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:38.087779 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:38.087841 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:38.088320 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:38.088322 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:38.088324 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:38.088391 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:38.088393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.089284 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.089285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:38.090684 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:38.090686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.091535 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.091536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:38.091657 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.091670 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:36:38.091765 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 06:36:38.091835 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":41,"thread_start_us":91,"threads_started":1}
I20260806 06:36:38.091954 13251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.087141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:38683 (local address 127.0.0.1:34502)
0806 06:36:38.087280 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:38.087327 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:38.087400 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:38.087980 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:38.087983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:38.087986 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:38.088100 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:38.088103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.089498 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.089499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:38.090507 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:38.090510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.091533 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.091535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:38.091639 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.091670 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 06:36:38.091795 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 06:36:38.091835 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":15,"thread_start_us":82,"threads_started":1}
I20260806 06:36:38.092375 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.087330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.29.142.129:38683)
0806 06:36:38.087801 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:38.087805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:38.087816 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:36:38.087859 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:38.087861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:38.087879 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:38.088043 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:38.088167 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.089346 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.090612 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.091389 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.091487 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.091558 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:38.092114 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 06:36:38.092288 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":116,"threads_started":1}
I20260806 06:36:38.093075 12972 tablet_service.cc:1467] Processing CreateTablet for tablet cde4d183eba9449498991fef21818c38 (DEFAULT_TABLE table=test-table [id=522d16e3525145858183480f94e41913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:38.093062 12795 tablet_service.cc:1467] Processing CreateTablet for tablet cde4d183eba9449498991fef21818c38 (DEFAULT_TABLE table=test-table [id=522d16e3525145858183480f94e41913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:38.093492 12795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde4d183eba9449498991fef21818c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:38.093502 12972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde4d183eba9449498991fef21818c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:38.096935 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.087186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:37881 (local address 127.0.0.1:33672)
0806 06:36:38.087548 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:38.087557 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:38.087610 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:38.087980 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:38.087983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:38.087990 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:38.088209 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:38.088210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.089467 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.089468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:38.090676 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:38.090679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.096603 (+ 5924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.096605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:38.096716 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.096723 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:36:38.096780 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:36:38.096836 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":42,"thread_start_us":111,"threads_started":1}
I20260806 06:36:38.097072 13262 tablet_bootstrap.cc:492] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e: Bootstrap starting.
I20260806 06:36:38.097474 13255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.087559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33672 (local address 127.29.142.131:37881)
0806 06:36:38.087773 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:38.087777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:38.087786 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:36:38.087857 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:38.087859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:38.087876 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:38.087963 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:38.088292 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.089370 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.091620 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.092369 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.097161 (+ 4792us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.097265 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:38.097301 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 06:36:38.097363 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":108,"threads_started":1}
I20260806 06:36:38.097927 13262 tablet_bootstrap.cc:654] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:38.098224 13150 tablet_service.cc:1467] Processing CreateTablet for tablet cde4d183eba9449498991fef21818c38 (DEFAULT_TABLE table=test-table [id=522d16e3525145858183480f94e41913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:36:38.098541 13262 log.cc:824] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:38.098614 13150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cde4d183eba9449498991fef21818c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:36:38.099634 13262 tablet_bootstrap.cc:492] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e: No bootstrap required, opened a new log
I20260806 06:36:38.099717 13262 ts_tablet_manager.cc:1397] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 06:36:38.100015 13259 tablet_bootstrap.cc:492] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930: Bootstrap starting.
I20260806 06:36:38.101238 13259 tablet_bootstrap.cc:654] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:38.102209 13259 log.cc:824] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:38.102474 13262 raft_consensus.cc:348] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26645061ced64ece9bb333be48756930" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38683 } } peers { permanent_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35087 } } peers { permanent_uuid: "3244c385d5be44aebca188dddc19f6cb" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37881 } }
I20260806 06:36:38.102705 13262 raft_consensus.cc:374] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:38.102777 13262 raft_consensus.cc:729] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1a7ab672db4aa6bc6d1f892f33965e, State: Initialized, Role: FOLLOWER
I20260806 06:36:38.102928 13262 consensus_queue.cc:260] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26645061ced64ece9bb333be48756930" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38683 } } peers { permanent_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35087 } } peers { permanent_uuid: "3244c385d5be44aebca188dddc19f6cb" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37881 } }
I20260806 06:36:38.102972 13264 tablet_bootstrap.cc:492] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb: Bootstrap starting.
I20260806 06:36:38.103267 13262 ts_tablet_manager.cc:1428] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:36:38.103410 13061 heartbeater.cc:503] Master 127.29.142.190:39475 was elected leader, sending a full tablet report...
I20260806 06:36:38.103598 13259 tablet_bootstrap.cc:492] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930: No bootstrap required, opened a new log
I20260806 06:36:38.103674 13259 ts_tablet_manager.cc:1397] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 06:36:38.103690 13264 tablet_bootstrap.cc:654] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb: Neither blocks nor log segments found. Creating new log.
I20260806 06:36:38.104281 13264 log.cc:824] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb: Log is configured to *not* fsync() on all Append() calls
I20260806 06:36:38.105098 13264 tablet_bootstrap.cc:492] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb: No bootstrap required, opened a new log
I20260806 06:36:38.105186 13264 ts_tablet_manager.cc:1397] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:36:38.105721 13259 raft_consensus.cc:348] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26645061ced64ece9bb333be48756930" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38683 } } peers { permanent_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35087 } } peers { permanent_uuid: "3244c385d5be44aebca188dddc19f6cb" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37881 } }
I20260806 06:36:38.105902 13259 raft_consensus.cc:374] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:38.105940 13259 raft_consensus.cc:729] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26645061ced64ece9bb333be48756930, State: Initialized, Role: FOLLOWER
I20260806 06:36:38.106056 13259 consensus_queue.cc:260] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26645061ced64ece9bb333be48756930" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38683 } } peers { permanent_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35087 } } peers { permanent_uuid: "3244c385d5be44aebca188dddc19f6cb" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37881 } }
I20260806 06:36:38.106403 12883 heartbeater.cc:503] Master 127.29.142.190:39475 was elected leader, sending a full tablet report...
I20260806 06:36:38.106632 13259 ts_tablet_manager.cc:1428] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:36:38.107311 13264 raft_consensus.cc:348] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26645061ced64ece9bb333be48756930" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38683 } } peers { permanent_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35087 } } peers { permanent_uuid: "3244c385d5be44aebca188dddc19f6cb" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37881 } }
I20260806 06:36:38.107474 13264 raft_consensus.cc:374] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:36:38.107522 13264 raft_consensus.cc:729] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3244c385d5be44aebca188dddc19f6cb, State: Initialized, Role: FOLLOWER
I20260806 06:36:38.107640 13264 consensus_queue.cc:260] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26645061ced64ece9bb333be48756930" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38683 } } peers { permanent_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35087 } } peers { permanent_uuid: "3244c385d5be44aebca188dddc19f6cb" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37881 } }
I20260806 06:36:38.107924 13264 ts_tablet_manager.cc:1428] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:36:38.108016 13238 heartbeater.cc:503] Master 127.29.142.190:39475 was elected leader, sending a full tablet report...
W20260806 06:36:38.131616 13062 tablet.cc:2369] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:36:38.183462 13268 raft_consensus.cc:482] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:36:38.183641 13268 raft_consensus.cc:504] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26645061ced64ece9bb333be48756930" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38683 } } peers { permanent_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35087 } } peers { permanent_uuid: "3244c385d5be44aebca188dddc19f6cb" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37881 } }
I20260806 06:36:38.184099 13268 leader_election.cc:288] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26645061ced64ece9bb333be48756930 (127.29.142.129:38683), 3244c385d5be44aebca188dddc19f6cb (127.29.142.131:37881)
I20260806 06:36:38.188594 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.184426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:40919 (local address 127.29.142.129:38683)
0806 06:36:38.184479 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:38.184480 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:38.184693 (+ 213us) server_negotiation.cc:407] Connection header received
0806 06:36:38.184773 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:38.184774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:38.184787 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:38.184850 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:38.185009 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.185721 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.187424 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.188118 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.188232 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.188242 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:38.188465 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 06:36:38.188510 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:36:38.188596 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.184345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:37881 (local address 127.29.142.130:37921)
0806 06:36:38.184452 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:38.184466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:38.184515 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:38.184752 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:38.184753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:38.184755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:38.184848 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:38.184851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.185842 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.185843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:38.187273 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:38.187279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.188266 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.188268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:38.188364 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.188366 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:36:38.188433 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:36:38.188497 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":82}
I20260806 06:36:38.188594 13255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.184487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:37921 (local address 127.29.142.131:37881)
0806 06:36:38.184544 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:38.184547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:38.184556 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:36:38.184590 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:38.184591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:38.184601 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:38.184692 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:36:38.184939 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.185730 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.187423 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.188137 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.188231 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.188242 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:38.188464 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 06:36:38.188510 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:36:38.188603 13272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.184356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:38683 (local address 127.29.142.130:40919)
0806 06:36:38.184587 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:38.184597 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:36:38.184653 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:36:38.184867 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:36:38.184868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:36:38.184870 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:36:38.184933 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:38.184935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.185842 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.185843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:38.187284 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:36:38.187286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.188266 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.188268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:36:38.188361 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.188366 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:36:38.188432 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:36:38.188497 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":112,"thread_start_us":151,"threads_started":1}
I20260806 06:36:38.189131 12817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde4d183eba9449498991fef21818c38" candidate_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26645061ced64ece9bb333be48756930" is_pre_election: true
I20260806 06:36:38.189177 13182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde4d183eba9449498991fef21818c38" candidate_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3244c385d5be44aebca188dddc19f6cb" is_pre_election: true
I20260806 06:36:38.189342 12817 raft_consensus.cc:2454] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1a7ab672db4aa6bc6d1f892f33965e in term 0.
I20260806 06:36:38.189359 13182 raft_consensus.cc:2454] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1a7ab672db4aa6bc6d1f892f33965e in term 0.
I20260806 06:36:38.189626 12914 leader_election.cc:302] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3244c385d5be44aebca188dddc19f6cb, 3f1a7ab672db4aa6bc6d1f892f33965e; no voters:
I20260806 06:36:38.189774 13268 raft_consensus.cc:2781] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:36:38.189836 13268 raft_consensus.cc:482] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:36:38.189873 13268 raft_consensus.cc:3037] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:38.190712 13268 raft_consensus.cc:504] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26645061ced64ece9bb333be48756930" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38683 } } peers { permanent_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35087 } } peers { permanent_uuid: "3244c385d5be44aebca188dddc19f6cb" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37881 } }
I20260806 06:36:38.190891 13268 leader_election.cc:288] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [CANDIDATE]: Term 1 election: Requested vote from peers 26645061ced64ece9bb333be48756930 (127.29.142.129:38683), 3244c385d5be44aebca188dddc19f6cb (127.29.142.131:37881)
I20260806 06:36:38.191061 13182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde4d183eba9449498991fef21818c38" candidate_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3244c385d5be44aebca188dddc19f6cb"
I20260806 06:36:38.191087 12817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cde4d183eba9449498991fef21818c38" candidate_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26645061ced64ece9bb333be48756930"
I20260806 06:36:38.191190 13182 raft_consensus.cc:3037] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:38.191190 12817 raft_consensus.cc:3037] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:36:38.191992 12817 raft_consensus.cc:2454] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1a7ab672db4aa6bc6d1f892f33965e in term 1.
I20260806 06:36:38.192106 13182 raft_consensus.cc:2454] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1a7ab672db4aa6bc6d1f892f33965e in term 1.
I20260806 06:36:38.192199 12911 leader_election.cc:302] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26645061ced64ece9bb333be48756930, 3f1a7ab672db4aa6bc6d1f892f33965e; no voters:
I20260806 06:36:38.192317 13268 raft_consensus.cc:2781] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:36:38.192538 13268 raft_consensus.cc:686] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 1 LEADER]: Becoming Leader. State: Replica: 3f1a7ab672db4aa6bc6d1f892f33965e, State: Running, Role: LEADER
I20260806 06:36:38.192698 13268 consensus_queue.cc:237] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26645061ced64ece9bb333be48756930" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38683 } } peers { permanent_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35087 } } peers { permanent_uuid: "3244c385d5be44aebca188dddc19f6cb" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37881 } }
I20260806 06:36:38.193754 12622 catalog_manager.cc:5495] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f1a7ab672db4aa6bc6d1f892f33965e (127.29.142.130). New cstate: current_term: 1 leader_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26645061ced64ece9bb333be48756930" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f1a7ab672db4aa6bc6d1f892f33965e" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3244c385d5be44aebca188dddc19f6cb" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37881 } health_report { overall_health: UNKNOWN } } }
I20260806 06:36:38.210353 13258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.207870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54084 (local address 127.29.142.130:35087)
0806 06:36:38.207932 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:36:38.207934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:36:38.207950 (+ 16us) server_negotiation.cc:407] Connection header received
0806 06:36:38.207989 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:36:38.207990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:36:38.208008 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:36:38.208071 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:36:38.208258 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.208772 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.209498 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:36:38.209683 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:36:38.209742 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:36:38.209888 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:36:38.209994 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:36:38.210059 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:36:38.210234 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 06:36:38.210256 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:36:38.212514 13182 raft_consensus.cc:1261] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb [term 1 FOLLOWER]: Refusing update from remote peer 3f1a7ab672db4aa6bc6d1f892f33965e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:38.212543 12817 raft_consensus.cc:1261] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930 [term 1 FOLLOWER]: Refusing update from remote peer 3f1a7ab672db4aa6bc6d1f892f33965e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:36:38.212944 13273 consensus_queue.cc:1035] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3244c385d5be44aebca188dddc19f6cb" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 37881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:38.213083 13268 consensus_queue.cc:1035] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [LEADER]: Connected to new peer: Peer: permanent_uuid: "26645061ced64ece9bb333be48756930" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:36:38.215895 13277 mvcc.cc:205] Tried to move back new op lower bound from 7315448619875373056 to 7315448619798110208. Current Snapshot: MvccSnapshot[applied={T|T < 7315448619875373056}]
I20260806 06:36:38.219082 12621 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:38.219223 12621 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:36:38.220127 12621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04d044207f5744258617c6f57e40074b: Sending DeleteTablet for 3 replicas of tablet cde4d183eba9449498991fef21818c38
I20260806 06:36:38.220525 12795 tablet_service.cc:1514] Processing DeleteTablet for tablet cde4d183eba9449498991fef21818c38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:38 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:34502
I20260806 06:36:38.220561 12972 tablet_service.cc:1514] Processing DeleteTablet for tablet cde4d183eba9449498991fef21818c38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:38 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:54076
I20260806 06:36:38.220551 13150 tablet_service.cc:1514] Processing DeleteTablet for tablet cde4d183eba9449498991fef21818c38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:36:38 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:33672
I20260806 06:36:38.220907 13293 tablet_replica.cc:330] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb: stopping tablet replica
I20260806 06:36:38.220934 13290 tablet_replica.cc:330] T cde4d183eba9449498991fef21818c38 P 26645061ced64ece9bb333be48756930: stopping tablet replica
I20260806 06:36:38.221174 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 12703
I20260806 06:36:38.221036 13293 raft_consensus.cc:2229] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:36:38.244346 13293 raft_consensus.cc:2258] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:38.244665 13291 tablet_replica.cc:330] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e: stopping tablet replica
I20260806 06:36:38.244832 13291 raft_consensus.cc:2229] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:36:38.244990 13291 raft_consensus.cc:2258] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:36:38.245360 13293 ts_tablet_manager.cc:1905] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:38.245517 13291 ts_tablet_manager.cc:1905] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:36:38.247357 13291 ts_tablet_manager.cc:1918] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:38.247357 13293 ts_tablet_manager.cc:1918] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:36:38.247444 13293 log.cc:1196] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/cde4d183eba9449498991fef21818c38
I20260806 06:36:38.247444 13291 log.cc:1196] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/cde4d183eba9449498991fef21818c38
I20260806 06:36:38.247792 13291 ts_tablet_manager.cc:1939] T cde4d183eba9449498991fef21818c38 P 3f1a7ab672db4aa6bc6d1f892f33965e: Deleting consensus metadata
I20260806 06:36:38.247792 13293 ts_tablet_manager.cc:1939] T cde4d183eba9449498991fef21818c38 P 3244c385d5be44aebca188dddc19f6cb: Deleting consensus metadata
I20260806 06:36:38.248301 12605 catalog_manager.cc:4841] TS 3f1a7ab672db4aa6bc6d1f892f33965e (127.29.142.130:35087): tablet cde4d183eba9449498991fef21818c38 (table test-table [id=522d16e3525145858183480f94e41913]) successfully deleted
I20260806 06:36:38.248312 12607 catalog_manager.cc:4841] TS 3244c385d5be44aebca188dddc19f6cb (127.29.142.131:37881): tablet cde4d183eba9449498991fef21818c38 (table test-table [id=522d16e3525145858183480f94e41913]) successfully deleted
W20260806 06:36:38.250690 12605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:38683 (error 108)
I20260806 06:36:38.251359 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:36:38.251200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:38683 (local address 127.0.0.1:34516)
0806 06:36:38.251267 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 06:36:38.251315 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:38683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:36:38.251403 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 12890
W20260806 06:36:38.251523 12605 catalog_manager.cc:4584] TS 26645061ced64ece9bb333be48756930 (127.29.142.129:38683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cde4d183eba9449498991fef21818c38: Network error: Client connection negotiation failed: client connection to 127.29.142.129:38683: connect: Connection refused (error 111)
I20260806 06:36:38.263398 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 13067
I20260806 06:36:38.273566 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 12579
2026-08-06T06:36:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2260 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:36:38 dist-test-slave-3kk6 krb5kdc[13298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:36:38 dist-test-slave-3kk6 krb5kdc[13298](info): set up 2 sockets
Aug 06 06:36:38 dist-test-slave-3kk6 krb5kdc[13298](info): commencing operation
krb5kdc: starting...
W20260806 06:36:39.508081 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:36:39 dist-test-slave-3kk6 krb5kdc[13298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:36:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:36:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.142.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.142.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/rangeradmin_127.29.142.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/rangeradmin_127.29.142.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.142.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.142.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/rangerlookup_127.29.142.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/rangerlookup_127.29.142.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.142.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.142.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/HTTP_127.29.142.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/HTTP_127.29.142.148@KRBTEST.COM.keytab.
I20260806 06:36:39.693105 30266 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrotugL/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/postgres -l logfile start
2026-08-06 06:36:42.282 UTC [13337] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 06:36:42.283 UTC [13337] LOG: listening on IPv4 address "127.29.142.148", port 54985
2026-08-06 06:36:42.286 UTC [13337] LOG: listening on Unix socket "/tmp/.s.PGSQL.54985"
2026-08-06 06:36:42.323 UTC [13342] LOG: database system was shut down at 2026-08-06 06:36:40 UTC
2026-08-06 06:36:42.328 UTC [13337] LOG: database system is ready to accept connections
I20260806 06:36:43.445387 30266 mini_postgres.cc:96] Postgres bound to 54985
2026-08-06 06:36:43.453 UTC [13353] FATAL: database "slave" does not exist
127.29.142.148:54985 - accepting connections
I20260806 06:36:43.454331 30266 mini_ranger.cc:161] Starting Ranger...
I20260806 06:36:43.471565 30266 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 06:36:43.962208 30266 mini_ranger.cc:87] Created ranger Postgres database
I20260806 06:36:43.962316 30266 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-admin
2026-08-06 06:36:44,282 [I] DB FLAVOR :POSTGRES
2026-08-06 06:36:44,283 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:36:44,283 [I] Checking connection..
2026-08-06 06:36:44,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 06:36:44,555 [I] Checking connection passed.
2026-08-06 06:36:44,555 [I] --------- Verifying version history table ---------
2026-08-06 06:36:44,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:36:44,796 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:36:44,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:36:45,057 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:36:45,057 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 06:36:45,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 06:36:45.330 UTC [13440] WARNING: there is no transaction in progress
2026-08-06 06:36:45,339 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 06:36:45,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:36:45,614 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 06:36:45,614 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:36:45,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 06:36:45,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 06:36:46,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-3kk6.c.gcp-upstream.internal','N') ;"
2026-08-06 06:36:46,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:36:46,693 [I] Table x_portal_user does not exist in database ranger
2026-08-06 06:36:46,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:36:46,973 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 06:36:46,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:36:47,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:36:47,536 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 06:36:47,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 06:36:48.564 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.581 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.599 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.615 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.630 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.647 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.701 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.708 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.722 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.733 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.744 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.759 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.770 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.777 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.784 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:48.791 UTC [13581] WARNING: there is no transaction in progress
2026-08-06 06:36:49,482 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 06:36:49,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 06:36:49,743 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 06:36:49,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:36:49,996 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 06:36:49,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:36:50,265 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 06:36:50,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:36:50,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:36:50,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-3kk6.c.gcp-upstream.internal';"
2026-08-06 06:36:51,047 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 06:36:51,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 06:36:51,290 [I] DB_PATCHES have already been applied
I20260806 06:36:51.296689 30266 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrotugL/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 06:36:51.296768 30266 mini_ranger.cc:193] Using host: 127.29.142.148
I20260806 06:36:51.300773 30266 mini_ranger.cc:227] Ranger admin URL: http://127.29.142.148:49207
Aug 06, 2026 6:36:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:36:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:36:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:36:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:36:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:36:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:36:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 06:36:52.623114 30266 mini_ranger.cc:160] Time spent starting Ranger: real 9.169s user 0.005s sys 0.006s
Aug 06, 2026 6:37:14 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:37:14 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 06, 2026 6:37:14 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:37:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:37:14 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:37:15 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 06:37:15.731547 30266 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.142.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.142.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/rangerkms_127.29.142.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/rangerkms_127.29.142.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.142.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.142.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.142.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/HTTP_127.29.142.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.142.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/HTTP_127.29.142.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 06:37:15 dist-test-slave-3kk6 krb5kdc[13298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998235, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 06:37:15.823529 30266 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 06:37:15.835805 30266 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 06:37:16.753656 30266 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 06:37:16.753759 30266 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-kms
I20260806 06:37:16.753780 30266 mini_ranger_kms.cc:227] Using postgres at 127.29.142.148:54985
2026-08-06 06:37:17,062 [I] DB FLAVOR :POSTGRES
2026-08-06 06:37:17,062 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:37:17,062 [I] Checking connection
2026-08-06 06:37:17,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 06:37:17,338 [I] connection success
2026-08-06 06:37:17,339 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 06:37:17,339 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 06:37:17,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 06:37:17,599 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 06:37:17,599 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:37:17,599 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 06:37:17,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:54985/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 06:37:17,920 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 06:37:19.034072 30266 mini_ranger_kms.cc:310] Created kms service
I20260806 06:37:19.493688 30266 mini_ranger_kms.cc:326] Created kudu user
I20260806 06:37:19.540814 30266 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 06:37:19 dist-test-slave-3kk6 krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998235, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:37:19.973831 30266 mini_ranger_kms.cc:384] Added ranger policy
I20260806 06:37:19.973975 30266 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 06:37:19.973996 30266 mini_ranger_kms.cc:240] Using host: 127.29.142.148
I20260806 06:37:19.978991 30266 mini_ranger_kms.cc:276] Ranger KMS PID: 13838
I20260806 06:37:19.979143 30266 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.142.148:43889
Aug 06, 2026 6:37:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:37:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:37:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:37:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:37:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:37:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 06:37:22 dist-test-slave-3kk6 krb5kdc[13298](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998242, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.142.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:37:22.734023 30266 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.911s user 0.018s sys 0.274s
I20260806 06:37:22.734130 30266 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 06:37:22.734215 30266 mini_ranger_kms.cc:401] 127.29.142.148:43889/kms/v1/keys
Aug 06 06:37:23 dist-test-slave-3kk6 krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998242, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.142.148@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
Aug 06 06:37:23 dist-test-slave-3kk6 krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998242, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.142.148@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
Aug 06 06:37:25 dist-test-slave-3kk6 krb5kdc[13298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:37:25.975848 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:34109
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:34109
--ranger_config_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.142.148:43889/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:37:26.069126 13898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:37:26.069372 13898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:37:26.069396 13898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:37:26.070588 13898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:37:26.070693 13898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:37:26.070722 13898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:37:26.070735 13898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:37:26.070748 13898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:37:26.072525 13898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.142.148:43889/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:34109
--ranger_config_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:34109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:37:26.072814 13898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:37:26.073086 13898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:37:26.078568 13904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:37:26.078702 13905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:37:26.078568 13907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:37:26.078867 13898 server_base.cc:1034] running on GCE node
I20260806 06:37:26.079203 13898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:37:26.079579 13898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:37:26.080735 13898 hybrid_clock.cc:648] HybridClock initialized: now 1785998246080702 us; error 48 us; skew 500 ppm
Aug 06 06:37:26 dist-test-slave-3kk6 krb5kdc[13298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998246, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:37:26.087677 13898 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:37:26.088199 13898 webserver.cc:460] Webserver started at http://127.29.142.190:36829/ using document root <none> and password file <none>
I20260806 06:37:26.088493 13898 fs_manager.cc:359] Metadata directory not provided
I20260806 06:37:26.088573 13898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:37:26.088747 13898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:37:26 dist-test-slave-3kk6 krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998246, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:37:26.178722 13898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "96c6dd0e723b44e8a5bcbfc2dbb873a8"
format_stamp: "Formatted at 2026-08-06 06:37:26 on dist-test-slave-3kk6"
server_key: "11912ca0977b4bdca74dc73858b161b5"
server_key_iv: "936ef5cad7fb1fc83388182b04eb6706"
server_key_version: "kuduclusterkey@0"
I20260806 06:37:26.179199 13898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "96c6dd0e723b44e8a5bcbfc2dbb873a8"
format_stamp: "Formatted at 2026-08-06 06:37:26 on dist-test-slave-3kk6"
server_key: "11912ca0977b4bdca74dc73858b161b5"
server_key_iv: "936ef5cad7fb1fc83388182b04eb6706"
server_key_version: "kuduclusterkey@0"
I20260806 06:37:26.181150 13898 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 06:37:26.205096 13917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:37:26.205391 13898 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260806 06:37:26.205547 13898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "96c6dd0e723b44e8a5bcbfc2dbb873a8"
format_stamp: "Formatted at 2026-08-06 06:37:26 on dist-test-slave-3kk6"
server_key: "11912ca0977b4bdca74dc73858b161b5"
server_key_iv: "936ef5cad7fb1fc83388182b04eb6706"
server_key_version: "kuduclusterkey@0"
I20260806 06:37:26.205667 13898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:37:26.243008 13898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:37:26.245232 13898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:37:26.245426 13898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:37:26.250528 13898 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:34109
I20260806 06:37:26.250597 13990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:34109 every 8 connection(s)
I20260806 06:37:26.250988 13898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:37:26.251735 13991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:37:26.254199 13991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8: Bootstrap starting.
I20260806 06:37:26.254977 13991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8: Neither blocks nor log segments found. Creating new log.
I20260806 06:37:26.255373 13991 log.cc:824] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8: Log is configured to *not* fsync() on all Append() calls
I20260806 06:37:26.255815 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 13898
I20260806 06:37:26.255892 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:37:26.256091 30266 external_mini_cluster.cc:1428] Setting key 3bbb068abd5161f68d67ed12729b4b9f
I20260806 06:37:26.256412 13991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8: No bootstrap required, opened a new log
I20260806 06:37:26.259722 13991 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c6dd0e723b44e8a5bcbfc2dbb873a8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 34109 } }
I20260806 06:37:26.259915 13991 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:37:26.259953 13991 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96c6dd0e723b44e8a5bcbfc2dbb873a8, State: Initialized, Role: FOLLOWER
I20260806 06:37:26.260078 13991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c6dd0e723b44e8a5bcbfc2dbb873a8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 34109 } }
I20260806 06:37:26.260149 13991 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:37:26.260176 13991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:37:26.260219 13991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:37:26.261085 13991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c6dd0e723b44e8a5bcbfc2dbb873a8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 34109 } }
I20260806 06:37:26.261229 13991 leader_election.cc:302] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96c6dd0e723b44e8a5bcbfc2dbb873a8; no voters:
I20260806 06:37:26.261404 13991 leader_election.cc:288] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:37:26.261480 13998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:37:26.261642 13998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [term 1 LEADER]: Becoming Leader. State: Replica: 96c6dd0e723b44e8a5bcbfc2dbb873a8, State: Running, Role: LEADER
I20260806 06:37:26.261688 13991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:37:26.261749 13998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c6dd0e723b44e8a5bcbfc2dbb873a8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 34109 } }
I20260806 06:37:26.262218 13998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96c6dd0e723b44e8a5bcbfc2dbb873a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c6dd0e723b44e8a5bcbfc2dbb873a8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 34109 } } }
I20260806 06:37:26.262313 13998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:37:26.262920 13998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96c6dd0e723b44e8a5bcbfc2dbb873a8. Latest consensus state: current_term: 1 leader_uuid: "96c6dd0e723b44e8a5bcbfc2dbb873a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96c6dd0e723b44e8a5bcbfc2dbb873a8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 34109 } } }
I20260806 06:37:26.263067 13998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:37:26.264071 13991 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:37:26.264287 14004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 06:37:26 dist-test-slave-3kk6 krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:37:26.267370 14004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:37:26.273149 13997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:26.256739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.29.142.190:34109)
0806 06:37:26.256989 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 06:37:26.257001 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 06:37:26.257038 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:37:26.257948 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:37:26.257958 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:37:26.258027 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:37:26.258188 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:37:26.258890 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:26.260412 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:26.261176 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:26.261860 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:26.261985 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:26.267027 (+ 5042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:37:26.267113 (+ 86us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:37:26.267113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:37:26.267357 (+ 244us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:37:26.271067 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:37:26.271565 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:37:26.271568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:37:26.271569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:37:26.271730 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:37:26.271997 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:37:26.271998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:37:26.271999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:37:26.272396 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:37:26.272612 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 06:37:26.272705 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 06:37:26.272877 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":122,"threads_started":1}
I20260806 06:37:26.275152 14004 catalog_manager.cc:1347] Generated new cluster ID: e3eb10c6fb7049e7b3c785fbf6af5a0f
I20260806 06:37:26.275223 14004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:37:26.310796 14004 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:37:26.311693 14004 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:37:26.315625 14004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8: Generated new TSK 0
I20260806 06:37:26.315824 14004 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:37:27.078056 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.142.148:43889/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:34109
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:37:27.179641 14047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:37:27.179893 14047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:37:27.179937 14047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:37:27.181134 14047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:37:27.181210 14047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:37:27.181259 14047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:37:27.183030 14047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.142.148:43889/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:34109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:37:27.183305 14047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:37:27.183630 14047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:37:27.189760 14054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:37:27.189760 14055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:37:27.190276 14057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:37:27.190521 14047 server_base.cc:1034] running on GCE node
I20260806 06:37:27.190783 14047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:37:27.191064 14047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:37:27.192251 14047 hybrid_clock.cc:648] HybridClock initialized: now 1785998247192236 us; error 28 us; skew 500 ppm
Aug 06 06:37:27 dist-test-slave-3kk6 krb5kdc[13298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:37:27.195505 14047 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:37:27.196138 14047 webserver.cc:460] Webserver started at http://127.29.142.129:46265/ using document root <none> and password file <none>
I20260806 06:37:27.196478 14047 fs_manager.cc:359] Metadata directory not provided
I20260806 06:37:27.196559 14047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:37:27.196708 14047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:37:27 dist-test-slave-3kk6 krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:37:27.213002 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "4be78fe3632548d791c101a614ac3f7b"
format_stamp: "Formatted at 2026-08-06 06:37:27 on dist-test-slave-3kk6"
server_key: "acadf8acac306575d6d54d0737510500"
server_key_iv: "44395617cf699cedbeccb856914ce13c"
server_key_version: "kuduclusterkey@0"
I20260806 06:37:27.213572 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "4be78fe3632548d791c101a614ac3f7b"
format_stamp: "Formatted at 2026-08-06 06:37:27 on dist-test-slave-3kk6"
server_key: "acadf8acac306575d6d54d0737510500"
server_key_iv: "44395617cf699cedbeccb856914ce13c"
server_key_version: "kuduclusterkey@0"
I20260806 06:37:27.216295 14047 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260806 06:37:27.231601 14066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:37:27.231878 14047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:37:27.231985 14047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "4be78fe3632548d791c101a614ac3f7b"
format_stamp: "Formatted at 2026-08-06 06:37:27 on dist-test-slave-3kk6"
server_key: "acadf8acac306575d6d54d0737510500"
server_key_iv: "44395617cf699cedbeccb856914ce13c"
server_key_version: "kuduclusterkey@0"
I20260806 06:37:27.232090 14047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:37:27.247851 14047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:37:27.253186 14047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:37:27.253417 14047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:37:27.253726 14047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:37:27.254276 14047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:37:27.254428 14047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:37:27.254549 14047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:37:27.254633 14047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:37:27.266105 14047 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:41767
I20260806 06:37:27.267204 14047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:37:27.274966 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 14047
I20260806 06:37:27.275090 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:37:27.275305 30266 external_mini_cluster.cc:1428] Setting key 8687d286861a4f5ffcff672d1d7b2f2a
I20260806 06:37:27.269632 14222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:41767 every 8 connection(s)
Aug 06 06:37:27 dist-test-slave-3kk6 krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:37:27.283912 14228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.270328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:34109 (local address 127.29.142.129:51751)
0806 06:37:27.271161 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0806 06:37:27.271220 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 06:37:27.271967 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:37:27.272462 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:37:27.272467 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:37:27.272754 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:37:27.273042 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:27.273048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.274120 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.274122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:27.276340 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:27.276343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.276906 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.276908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:27.277056 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.277745 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:37:27.277756 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:37:27.279654 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:37:27.281973 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:37:27.281982 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:37:27.281985 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:37:27.282304 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:37:27.282713 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:37:27.282714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:37:27.282714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:37:27.282860 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:37:27.283165 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:37:27.283172 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:37:27.283374 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 06:37:27.283483 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 06:37:27.283652 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":181,"threads_started":1}
I20260806 06:37:27.284534 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.270724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:51751 (local address 127.29.142.190:34109)
0806 06:37:27.272180 (+ 1456us) server_negotiation.cc:206] Beginning negotiation
0806 06:37:27.272183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:37:27.272192 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:37:27.272229 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:37:27.272230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:37:27.272243 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:37:27.272318 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:37:27.273167 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.274002 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.276442 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.276764 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.276831 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.279784 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:37:27.279786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:37:27.279786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:37:27.279794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:37:27.281840 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:37:27.282406 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:37:27.282407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:37:27.282407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:37:27.282473 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:37:27.282936 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:37:27.282937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:37:27.282937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:37:27.283081 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:37:27.283139 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:37:27.284350 (+ 1211us) server_negotiation.cc:299] Negotiation successful
0806 06:37:27.284439 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1413,"thread_start_us":144,"threads_started":1}
I20260806 06:37:27.285384 14224 heartbeater.cc:347] Connected to a master server at 127.29.142.190:34109
I20260806 06:37:27.285493 14224 heartbeater.cc:464] Registering TS with master...
I20260806 06:37:27.285699 14224 heartbeater.cc:511] Master 127.29.142.190:34109 requested a full tablet report, sending...
I20260806 06:37:27.286249 13946 ts_manager.cc:194] Registered new tserver with Master: 4be78fe3632548d791c101a614ac3f7b (127.29.142.129:41767)
I20260806 06:37:27.287124 13946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:51751
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:37:27.317476 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.142.148:43889/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:34109
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:37:27.457160 14233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:37:27.457556 14233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:37:27.457649 14233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:37:27.459650 14233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:37:27.459793 14233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:37:27.459906 14233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:37:27.462486 14233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.142.148:43889/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:34109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:37:27.463032 14233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:37:27.463553 14233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:37:27.471546 14242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:37:27.471738 14244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:37:27.472082 14241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:37:27.473385 14233 server_base.cc:1034] running on GCE node
I20260806 06:37:27.473628 14233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:37:27.473954 14233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:37:27.475132 14233 hybrid_clock.cc:648] HybridClock initialized: now 1785998247475106 us; error 53 us; skew 500 ppm
Aug 06 06:37:27 dist-test-slave-3kk6 krb5kdc[13298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:37:27.478961 14233 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:37:27.479540 14233 webserver.cc:460] Webserver started at http://127.29.142.130:39731/ using document root <none> and password file <none>
I20260806 06:37:27.479861 14233 fs_manager.cc:359] Metadata directory not provided
I20260806 06:37:27.479930 14233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:37:27.480065 14233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:37:27 dist-test-slave-3kk6 krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
Aug 06 06:37:27 dist-test-slave-3kk6 krb5kdc[13298](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:37:27.494386 14233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "9977c7e74bd9469d8a700ac890de45a8"
format_stamp: "Formatted at 2026-08-06 06:37:27 on dist-test-slave-3kk6"
server_key: "07fc1f6e99a9e9b6c6d8ccc905571a67"
server_key_iv: "dd1a046a6933909ed8f0ba7866e7ecf0"
server_key_version: "kuduclusterkey@0"
I20260806 06:37:27.494874 14233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "9977c7e74bd9469d8a700ac890de45a8"
format_stamp: "Formatted at 2026-08-06 06:37:27 on dist-test-slave-3kk6"
server_key: "07fc1f6e99a9e9b6c6d8ccc905571a67"
server_key_iv: "dd1a046a6933909ed8f0ba7866e7ecf0"
server_key_version: "kuduclusterkey@0"
I20260806 06:37:27.499112 14233 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.001s
I20260806 06:37:27.517984 14258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:37:27.518285 14233 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260806 06:37:27.518399 14233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "9977c7e74bd9469d8a700ac890de45a8"
format_stamp: "Formatted at 2026-08-06 06:37:27 on dist-test-slave-3kk6"
server_key: "07fc1f6e99a9e9b6c6d8ccc905571a67"
server_key_iv: "dd1a046a6933909ed8f0ba7866e7ecf0"
server_key_version: "kuduclusterkey@0"
I20260806 06:37:27.518519 14233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:37:27.540160 14233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:37:27.542461 14233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:37:27.542696 14233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:37:27.542980 14233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:37:27.543488 14233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:37:27.543557 14233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:37:27.543628 14233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:37:27.543659 14233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:37:27.551102 14233 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:34125
I20260806 06:37:27.551813 14233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:37:27.559044 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 14233
I20260806 06:37:27.559176 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:37:27.559231 14413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:34125 every 8 connection(s)
I20260806 06:37:27.559423 30266 external_mini_cluster.cc:1428] Setting key 2dd63544b383c39cecf2e6e32f7d304d
Aug 06 06:37:27 dist-test-slave-3kk6 krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
I20260806 06:37:27.572877 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.560005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:45249 (local address 127.29.142.190:34109)
0806 06:37:27.560061 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:37:27.560063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:37:27.560387 (+ 324us) server_negotiation.cc:407] Connection header received
0806 06:37:27.561260 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:37:27.561261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:37:27.561297 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:37:27.561331 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:37:27.562586 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.563614 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.565337 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.565724 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.565784 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.569145 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:37:27.569147 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:37:27.569147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:37:27.569167 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:37:27.571129 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:37:27.571741 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:37:27.571743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:37:27.571743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:37:27.571809 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:37:27.572141 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:37:27.572143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:37:27.572143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:37:27.572304 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:37:27.572343 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:37:27.572677 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 06:37:27.572771 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:37:27.573184 14417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.560009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:34109 (local address 127.29.142.130:45249)
0806 06:37:27.560241 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 06:37:27.560290 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:37:27.561114 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:37:27.561598 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:37:27.561603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:37:27.561906 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:37:27.562446 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:27.562461 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.563818 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.563820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:27.565222 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:27.565226 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.565900 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.565901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:27.566070 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.566824 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:37:27.566837 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:37:27.569010 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:37:27.571258 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:37:27.571267 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:37:27.571270 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:37:27.571610 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:37:27.571896 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:37:27.571897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:37:27.571898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:37:27.572054 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:37:27.572386 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:37:27.572391 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:37:27.572523 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 06:37:27.572647 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 06:37:27.572849 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":142,"threads_started":1}
I20260806 06:37:27.573873 14414 heartbeater.cc:347] Connected to a master server at 127.29.142.190:34109
I20260806 06:37:27.573993 14414 heartbeater.cc:464] Registering TS with master...
I20260806 06:37:27.574205 14414 heartbeater.cc:511] Master 127.29.142.190:34109 requested a full tablet report, sending...
I20260806 06:37:27.574707 13946 ts_manager.cc:194] Registered new tserver with Master: 9977c7e74bd9469d8a700ac890de45a8 (127.29.142.130:34125)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
I20260806 06:37:27.575269 13946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:45249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:37:27.598414 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.142.148:43889/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:34109
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:37:27.741703 14421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:37:27.742125 14421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:37:27.742242 14421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:37:27.744114 14421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:37:27.744364 14421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:37:27.744473 14421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:37:27.746951 14421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.142.148:43889/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:34109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:37:27.747439 14421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:37:27.747809 14421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:37:27.755818 14429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:37:27.756172 14421 server_base.cc:1034] running on GCE node
W20260806 06:37:27.762828 14428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:37:27.763662 14431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:37:27.764056 14421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:37:27.764369 14421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:37:27.767653 14421 hybrid_clock.cc:648] HybridClock initialized: now 1785998247767631 us; error 43 us; skew 500 ppm
Aug 06 06:37:27 dist-test-slave-3kk6 krb5kdc[13298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:37:27.771955 14421 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:37:27.772701 14421 webserver.cc:460] Webserver started at http://127.29.142.131:43397/ using document root <none> and password file <none>
I20260806 06:37:27.773041 14421 fs_manager.cc:359] Metadata directory not provided
I20260806 06:37:27.773125 14421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:37:27.773299 14421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:37:27 dist-test-slave-3kk6 krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:37:27.787818 14421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "98a6ba7cb9ad41b2b33ef465a0d44250"
format_stamp: "Formatted at 2026-08-06 06:37:27 on dist-test-slave-3kk6"
server_key: "293d352cb86d8aa5bda4de805dc3df43"
server_key_iv: "93a1c333bca5d2d1f17decc3f4b707ea"
server_key_version: "kuduclusterkey@0"
I20260806 06:37:27.788589 14421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "98a6ba7cb9ad41b2b33ef465a0d44250"
format_stamp: "Formatted at 2026-08-06 06:37:27 on dist-test-slave-3kk6"
server_key: "293d352cb86d8aa5bda4de805dc3df43"
server_key_iv: "93a1c333bca5d2d1f17decc3f4b707ea"
server_key_version: "kuduclusterkey@0"
I20260806 06:37:27.799463 14421 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.000s sys 0.002s
I20260806 06:37:27.818811 14440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:37:27.819084 14421 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.001s
I20260806 06:37:27.819181 14421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "98a6ba7cb9ad41b2b33ef465a0d44250"
format_stamp: "Formatted at 2026-08-06 06:37:27 on dist-test-slave-3kk6"
server_key: "293d352cb86d8aa5bda4de805dc3df43"
server_key_iv: "93a1c333bca5d2d1f17decc3f4b707ea"
server_key_version: "kuduclusterkey@0"
I20260806 06:37:27.819294 14421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:37:27.848732 14421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:37:27.851509 14421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:37:27.851817 14421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:37:27.852267 14421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:37:27.852762 14421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:37:27.852883 14421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:37:27.852962 14421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:37:27.853044 14421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:37:27.868140 14421 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:44065
I20260806 06:37:27.868803 14421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:37:27.871277 14607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:44065 every 8 connection(s)
I20260806 06:37:27.872875 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 14421
I20260806 06:37:27.872993 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:37:27.873204 30266 external_mini_cluster.cc:1428] Setting key 03171f069247a08f978ef4aa77e9f569
Aug 06 06:37:27 dist-test-slave-3kk6 krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:37:27.885187 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.872103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:58007 (local address 127.29.142.190:34109)
0806 06:37:27.872162 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:37:27.872165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:37:27.873782 (+ 1617us) server_negotiation.cc:407] Connection header received
0806 06:37:27.874619 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:37:27.874620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:37:27.874633 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:37:27.874724 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:37:27.875626 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.876375 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.877810 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.878098 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.878179 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.880941 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:37:27.880942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:37:27.880942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:37:27.880951 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:37:27.882626 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:37:27.883149 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:37:27.883150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:37:27.883150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:37:27.883211 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:37:27.883503 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:37:27.883504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:37:27.883504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:37:27.883707 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:37:27.883794 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:37:27.884992 (+ 1198us) server_negotiation.cc:299] Negotiation successful
0806 06:37:27.885089 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:37:27.885466 14610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.872100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:34109 (local address 127.29.142.131:58007)
0806 06:37:27.873640 (+ 1540us) negotiation.cc:107] Waiting for socket to connect
0806 06:37:27.873690 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 06:37:27.874491 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:37:27.874814 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:37:27.874819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:37:27.875119 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:37:27.875498 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:27.875504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.876480 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.876481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:27.877707 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:27.877710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.878199 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.878201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:27.878349 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.879101 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:37:27.879112 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:37:27.880834 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:37:27.882755 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:37:27.882762 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:37:27.882765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:37:27.883058 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:37:27.883295 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:37:27.883296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:37:27.883296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:37:27.883423 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:37:27.883836 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:37:27.883844 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:37:27.884052 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 06:37:27.884945 (+ 893us) client_negotiation.cc:240] Negotiation successful
0806 06:37:27.885161 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1497,"thread_start_us":448,"threads_started":1}
I20260806 06:37:27.886051 14608 heartbeater.cc:347] Connected to a master server at 127.29.142.190:34109
I20260806 06:37:27.886165 14608 heartbeater.cc:464] Registering TS with master...
I20260806 06:37:27.886374 14608 heartbeater.cc:511] Master 127.29.142.190:34109 requested a full tablet report, sending...
I20260806 06:37:27.886871 13946 ts_manager.cc:194] Registered new tserver with Master: 98a6ba7cb9ad41b2b33ef465a0d44250 (127.29.142.131:44065)
I20260806 06:37:27.887409 13946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:58007
I20260806 06:37:27.895741 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:37:27.913130 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.904400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55800 (local address 127.29.142.190:34109)
0806 06:37:27.904459 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:37:27.904462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:37:27.904805 (+ 343us) server_negotiation.cc:407] Connection header received
0806 06:37:27.904936 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:37:27.904937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:37:27.904951 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:37:27.904987 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:37:27.905948 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.906768 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.907864 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.908143 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.908189 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.909929 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:37:27.909930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:37:27.909931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:37:27.909939 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:37:27.911803 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:37:27.912230 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:37:27.912232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:37:27.912232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:37:27.912303 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:37:27.912510 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:37:27.912511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:37:27.912511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:37:27.912643 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:37:27.912716 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:37:27.912943 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 06:37:27.913028 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:37:27.915555 13946 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:37:27.918040 13946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:37:27.932891 14626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.921873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:41767 (local address 127.0.0.1:48502)
0806 06:37:27.924500 (+ 2627us) negotiation.cc:107] Waiting for socket to connect
0806 06:37:27.924509 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:37:27.924554 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:37:27.928962 (+ 4408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:37:27.928963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:37:27.928966 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:37:27.929022 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:27.929024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.930230 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.930231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:27.931479 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:27.931484 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.932573 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.932574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:27.932660 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.932675 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:37:27.932747 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:37:27.932799 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2609,"thread_start_us":63,"threads_started":1}
I20260806 06:37:27.933414 14634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.922436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.29.142.129:41767)
0806 06:37:27.928704 (+ 6268us) server_negotiation.cc:206] Beginning negotiation
0806 06:37:27.928709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:37:27.928720 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:37:27.928758 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:37:27.928760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:37:27.928778 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:37:27.928877 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:37:27.929099 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.930106 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.931604 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.932394 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.932471 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.932534 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:37:27.933293 (+ 759us) server_negotiation.cc:299] Negotiation successful
0806 06:37:27.933342 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6223,"thread_start_us":60,"threads_started":1}
I20260806 06:37:27.939065 14123 tablet_service.cc:1467] Processing CreateTablet for tablet a06b39cf4f20490cbc68021c17de46b7 (DEFAULT_TABLE table=test-table [id=58e2a388e5684f858d5274ffe9f66fbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:37:27.939556 14123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a06b39cf4f20490cbc68021c17de46b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:37:27.939795 14631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.921624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:44065 (local address 127.0.0.1:45102)
0806 06:37:27.923754 (+ 2130us) negotiation.cc:107] Waiting for socket to connect
0806 06:37:27.923823 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 06:37:27.923896 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:37:27.924932 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:37:27.924935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:37:27.924938 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:37:27.925025 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:27.925027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.929210 (+ 4183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.929211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:27.930320 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:27.930323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.939472 (+ 9149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.939474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:27.939557 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.939564 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:37:27.939629 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:37:27.939695 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2089,"thread_start_us":85,"threads_started":1}
I20260806 06:37:27.940153 14630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.921746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45102 (local address 127.29.142.131:44065)
0806 06:37:27.924674 (+ 2928us) server_negotiation.cc:206] Beginning negotiation
0806 06:37:27.924678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:37:27.924688 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:37:27.924726 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:37:27.924728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:37:27.924743 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:37:27.924823 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:37:27.927391 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.928516 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.938300 (+ 9784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.939161 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.939332 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.939409 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:37:27.940011 (+ 602us) server_negotiation.cc:299] Negotiation successful
0806 06:37:27.940066 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2890,"thread_start_us":94,"threads_started":1}
I20260806 06:37:27.940804 14517 tablet_service.cc:1467] Processing CreateTablet for tablet a06b39cf4f20490cbc68021c17de46b7 (DEFAULT_TABLE table=test-table [id=58e2a388e5684f858d5274ffe9f66fbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:37:27.941152 14517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a06b39cf4f20490cbc68021c17de46b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:37:27.944658 14636 tablet_bootstrap.cc:492] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b: Bootstrap starting.
I20260806 06:37:27.945022 14637 tablet_bootstrap.cc:492] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250: Bootstrap starting.
I20260806 06:37:27.945555 14636 tablet_bootstrap.cc:654] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b: Neither blocks nor log segments found. Creating new log.
I20260806 06:37:27.946120 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.921624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:34125 (local address 127.0.0.1:54072)
0806 06:37:27.924101 (+ 2477us) negotiation.cc:107] Waiting for socket to connect
0806 06:37:27.924110 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:37:27.924158 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:37:27.925103 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:37:27.925103 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:37:27.925105 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:37:27.925131 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:27.925132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.926972 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.926973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:27.929099 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:27.929103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.945800 (+ 16697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.945802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:27.945904 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.945911 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:37:27.945983 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:37:27.946037 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2462,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260806 06:37:27.946825 14636 log.cc:824] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:37:27.947345 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:27.922187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54072 (local address 127.29.142.130:34125)
0806 06:37:27.924050 (+ 1863us) server_negotiation.cc:206] Beginning negotiation
0806 06:37:27.924055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:37:27.924210 (+ 155us) server_negotiation.cc:407] Connection header received
0806 06:37:27.924254 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:37:27.924256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:37:27.924273 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:37:27.924342 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:37:27.925568 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.926871 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.933119 (+ 6248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:27.945688 (+ 12569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:27.945811 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:27.945881 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:37:27.947201 (+ 1320us) server_negotiation.cc:299] Negotiation successful
0806 06:37:27.947259 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1824,"thread_start_us":111,"threads_started":1}
I20260806 06:37:27.948587 14324 tablet_service.cc:1467] Processing CreateTablet for tablet a06b39cf4f20490cbc68021c17de46b7 (DEFAULT_TABLE table=test-table [id=58e2a388e5684f858d5274ffe9f66fbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:37:27.948984 14324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a06b39cf4f20490cbc68021c17de46b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:37:27.953344 14641 tablet_bootstrap.cc:492] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8: Bootstrap starting.
I20260806 06:37:27.954509 14641 tablet_bootstrap.cc:654] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8: Neither blocks nor log segments found. Creating new log.
I20260806 06:37:27.954946 14641 log.cc:824] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8: Log is configured to *not* fsync() on all Append() calls
I20260806 06:37:27.956156 14636 tablet_bootstrap.cc:492] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b: No bootstrap required, opened a new log
I20260806 06:37:27.956295 14636 ts_tablet_manager.cc:1397] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260806 06:37:27.956389 14637 tablet_bootstrap.cc:654] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250: Neither blocks nor log segments found. Creating new log.
I20260806 06:37:27.959621 14637 log.cc:824] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250: Log is configured to *not* fsync() on all Append() calls
I20260806 06:37:27.960086 14636 raft_consensus.cc:348] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9977c7e74bd9469d8a700ac890de45a8" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34125 } } peers { permanent_uuid: "4be78fe3632548d791c101a614ac3f7b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41767 } } peers { permanent_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 44065 } }
I20260806 06:37:27.960393 14636 raft_consensus.cc:374] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:37:27.960502 14636 raft_consensus.cc:729] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4be78fe3632548d791c101a614ac3f7b, State: Initialized, Role: FOLLOWER
I20260806 06:37:27.960736 14636 consensus_queue.cc:260] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9977c7e74bd9469d8a700ac890de45a8" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34125 } } peers { permanent_uuid: "4be78fe3632548d791c101a614ac3f7b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41767 } } peers { permanent_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 44065 } }
I20260806 06:37:27.961148 14636 ts_tablet_manager.cc:1428] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 06:37:27.961373 14224 heartbeater.cc:503] Master 127.29.142.190:34109 was elected leader, sending a full tablet report...
I20260806 06:37:27.962285 14637 tablet_bootstrap.cc:492] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250: No bootstrap required, opened a new log
I20260806 06:37:27.962381 14637 ts_tablet_manager.cc:1397] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260806 06:37:27.963505 14641 tablet_bootstrap.cc:492] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8: No bootstrap required, opened a new log
I20260806 06:37:27.963573 14641 ts_tablet_manager.cc:1397] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260806 06:37:27.964603 14637 raft_consensus.cc:348] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9977c7e74bd9469d8a700ac890de45a8" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34125 } } peers { permanent_uuid: "4be78fe3632548d791c101a614ac3f7b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41767 } } peers { permanent_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 44065 } }
I20260806 06:37:27.964784 14637 raft_consensus.cc:374] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:37:27.964823 14637 raft_consensus.cc:729] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98a6ba7cb9ad41b2b33ef465a0d44250, State: Initialized, Role: FOLLOWER
I20260806 06:37:27.964944 14637 consensus_queue.cc:260] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9977c7e74bd9469d8a700ac890de45a8" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34125 } } peers { permanent_uuid: "4be78fe3632548d791c101a614ac3f7b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41767 } } peers { permanent_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 44065 } }
I20260806 06:37:27.965240 14637 ts_tablet_manager.cc:1428] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:37:27.965689 14641 raft_consensus.cc:348] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9977c7e74bd9469d8a700ac890de45a8" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34125 } } peers { permanent_uuid: "4be78fe3632548d791c101a614ac3f7b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41767 } } peers { permanent_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 44065 } }
I20260806 06:37:27.965894 14608 heartbeater.cc:503] Master 127.29.142.190:34109 was elected leader, sending a full tablet report...
I20260806 06:37:27.965911 14641 raft_consensus.cc:374] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:37:27.965955 14641 raft_consensus.cc:729] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9977c7e74bd9469d8a700ac890de45a8, State: Initialized, Role: FOLLOWER
I20260806 06:37:27.966087 14641 consensus_queue.cc:260] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9977c7e74bd9469d8a700ac890de45a8" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34125 } } peers { permanent_uuid: "4be78fe3632548d791c101a614ac3f7b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41767 } } peers { permanent_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 44065 } }
I20260806 06:37:27.966352 14641 ts_tablet_manager.cc:1428] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:37:27.966468 14414 heartbeater.cc:503] Master 127.29.142.190:34109 was elected leader, sending a full tablet report...
I20260806 06:37:28.010100 14646 raft_consensus.cc:482] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:37:28.010259 14646 raft_consensus.cc:504] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9977c7e74bd9469d8a700ac890de45a8" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34125 } } peers { permanent_uuid: "4be78fe3632548d791c101a614ac3f7b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41767 } } peers { permanent_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 44065 } }
I20260806 06:37:28.010771 14646 leader_election.cc:288] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9977c7e74bd9469d8a700ac890de45a8 (127.29.142.130:34125), 98a6ba7cb9ad41b2b33ef465a0d44250 (127.29.142.131:44065)
W20260806 06:37:28.021564 14227 tablet.cc:2369] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:37:28.022818 14630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:28.011945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:58879 (local address 127.29.142.131:44065)
0806 06:37:28.012031 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:37:28.012033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:37:28.014318 (+ 2285us) server_negotiation.cc:407] Connection header received
0806 06:37:28.014498 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:37:28.014499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:37:28.014511 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:37:28.014543 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:37:28.019299 (+ 4756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:28.019984 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:28.021851 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:28.022368 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:28.022422 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:28.022432 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:37:28.022692 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 06:37:28.022744 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 06:37:28.023705 14652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:28.011528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:44065 (local address 127.29.142.129:58879)
0806 06:37:28.014214 (+ 2686us) negotiation.cc:107] Waiting for socket to connect
0806 06:37:28.014223 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:37:28.014356 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:37:28.014798 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:37:28.014799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:37:28.014801 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:37:28.014903 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:28.014905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:28.020128 (+ 5223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:28.020129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:28.021170 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:28.021173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:28.022497 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:28.022497 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:28.022571 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:28.022574 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:37:28.023566 (+ 992us) client_negotiation.cc:240] Negotiation successful
0806 06:37:28.023617 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2661,"thread_start_us":100,"threads_started":1}
I20260806 06:37:28.025743 14543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a06b39cf4f20490cbc68021c17de46b7" candidate_uuid: "4be78fe3632548d791c101a614ac3f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250" is_pre_election: true
I20260806 06:37:28.025946 14543 raft_consensus.cc:2454] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4be78fe3632548d791c101a614ac3f7b in term 0.
I20260806 06:37:28.026193 14068 leader_election.cc:302] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4be78fe3632548d791c101a614ac3f7b, 98a6ba7cb9ad41b2b33ef465a0d44250; no voters:
I20260806 06:37:28.026335 14646 raft_consensus.cc:2781] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:37:28.026377 14646 raft_consensus.cc:482] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:37:28.026400 14646 raft_consensus.cc:3037] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:37:28.027725 14646 raft_consensus.cc:504] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9977c7e74bd9469d8a700ac890de45a8" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34125 } } peers { permanent_uuid: "4be78fe3632548d791c101a614ac3f7b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41767 } } peers { permanent_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 44065 } }
I20260806 06:37:28.027887 14646 leader_election.cc:288] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [CANDIDATE]: Term 1 election: Requested vote from peers 9977c7e74bd9469d8a700ac890de45a8 (127.29.142.130:34125), 98a6ba7cb9ad41b2b33ef465a0d44250 (127.29.142.131:44065)
I20260806 06:37:28.028332 14543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a06b39cf4f20490cbc68021c17de46b7" candidate_uuid: "4be78fe3632548d791c101a614ac3f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250"
I20260806 06:37:28.028422 14543 raft_consensus.cc:3037] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:37:28.029573 14543 raft_consensus.cc:2454] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4be78fe3632548d791c101a614ac3f7b in term 1.
I20260806 06:37:28.029865 14068 leader_election.cc:302] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4be78fe3632548d791c101a614ac3f7b, 98a6ba7cb9ad41b2b33ef465a0d44250; no voters:
I20260806 06:37:28.030010 14646 raft_consensus.cc:2781] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:37:28.030211 14646 raft_consensus.cc:686] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [term 1 LEADER]: Becoming Leader. State: Replica: 4be78fe3632548d791c101a614ac3f7b, State: Running, Role: LEADER
I20260806 06:37:28.030316 14646 consensus_queue.cc:237] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9977c7e74bd9469d8a700ac890de45a8" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34125 } } peers { permanent_uuid: "4be78fe3632548d791c101a614ac3f7b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41767 } } peers { permanent_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 44065 } }
I20260806 06:37:28.031375 13945 catalog_manager.cc:5495] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4be78fe3632548d791c101a614ac3f7b (127.29.142.129). New cstate: current_term: 1 leader_uuid: "4be78fe3632548d791c101a614ac3f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9977c7e74bd9469d8a700ac890de45a8" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4be78fe3632548d791c101a614ac3f7b" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 41767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 44065 } health_report { overall_health: UNKNOWN } } }
I20260806 06:37:28.032181 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:28.011894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:42133 (local address 127.29.142.130:34125)
0806 06:37:28.011994 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 06:37:28.011996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:37:28.014152 (+ 2156us) server_negotiation.cc:407] Connection header received
0806 06:37:28.014629 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:37:28.014630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:37:28.014641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:37:28.014712 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:37:28.015129 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:28.015886 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:28.024345 (+ 8459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:28.024894 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:28.024952 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:28.024962 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:37:28.032029 (+ 7067us) server_negotiation.cc:299] Negotiation successful
0806 06:37:28.032106 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 06:37:28.032897 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:28.011212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:34125 (local address 127.29.142.129:42133)
0806 06:37:28.014057 (+ 2845us) negotiation.cc:107] Waiting for socket to connect
0806 06:37:28.014073 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:37:28.014409 (+ 336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:37:28.014998 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:37:28.014998 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:37:28.015000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:37:28.015053 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:28.015054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:28.017826 (+ 2772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:28.017827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:28.019169 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:37:28.019173 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:37:28.025035 (+ 5862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:37:28.025036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:37:28.025117 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:28.025120 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:37:28.032705 (+ 7585us) client_negotiation.cc:240] Negotiation successful
0806 06:37:28.032779 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2821,"thread_start_us":150,"threads_started":1}
I20260806 06:37:28.035091 14351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a06b39cf4f20490cbc68021c17de46b7" candidate_uuid: "4be78fe3632548d791c101a614ac3f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9977c7e74bd9469d8a700ac890de45a8" is_pre_election: true
I20260806 06:37:28.035259 14351 raft_consensus.cc:2454] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4be78fe3632548d791c101a614ac3f7b in term 0.
I20260806 06:37:28.035387 14350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a06b39cf4f20490cbc68021c17de46b7" candidate_uuid: "4be78fe3632548d791c101a614ac3f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9977c7e74bd9469d8a700ac890de45a8"
I20260806 06:37:28.035459 14350 raft_consensus.cc:3037] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:37:28.036552 14350 raft_consensus.cc:2454] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4be78fe3632548d791c101a614ac3f7b in term 1.
I20260806 06:37:28.052673 14634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:28.049119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48508 (local address 127.29.142.129:41767)
0806 06:37:28.049161 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:37:28.049163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:37:28.049170 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:37:28.049213 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:37:28.049213 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:37:28.049236 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:37:28.049300 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:37:28.049523 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:28.050271 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:28.051431 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:37:28.051761 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:37:28.051843 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:37:28.052065 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:37:28.052206 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:37:28.052284 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:37:28.052537 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 06:37:28.052561 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:37:28.056411 14543 raft_consensus.cc:1261] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250 [term 1 FOLLOWER]: Refusing update from remote peer 4be78fe3632548d791c101a614ac3f7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:37:28.056915 14655 consensus_queue.cc:1035] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "98a6ba7cb9ad41b2b33ef465a0d44250" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 44065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:37:28.060072 14416 tablet.cc:2369] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:37:28.062851 14350 raft_consensus.cc:1261] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8 [term 1 FOLLOWER]: Refusing update from remote peer 4be78fe3632548d791c101a614ac3f7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:37:28.063180 14655 consensus_queue.cc:1035] T a06b39cf4f20490cbc68021c17de46b7 P 4be78fe3632548d791c101a614ac3f7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9977c7e74bd9469d8a700ac890de45a8" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:37:28.066110 14658 mvcc.cc:205] Tried to move back new op lower bound from 7315448824035643392 to 7315448823932690432. Current Snapshot: MvccSnapshot[applied={T|T < 7315448824035643392}]
I20260806 06:37:28.068537 14659 mvcc.cc:205] Tried to move back new op lower bound from 7315448824035643392 to 7315448823932690432. Current Snapshot: MvccSnapshot[applied={T|T < 7315448824035643392}]
I20260806 06:37:28.069058 13945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:37:28.069170 13945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:37:28.069978 13945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96c6dd0e723b44e8a5bcbfc2dbb873a8: Sending DeleteTablet for 3 replicas of tablet a06b39cf4f20490cbc68021c17de46b7
I20260806 06:37:28.070348 14517 tablet_service.cc:1514] Processing DeleteTablet for tablet a06b39cf4f20490cbc68021c17de46b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:37:28 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:45102
I20260806 06:37:28.070576 14324 tablet_service.cc:1514] Processing DeleteTablet for tablet a06b39cf4f20490cbc68021c17de46b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:37:28 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:54072
I20260806 06:37:28.071050 14123 tablet_service.cc:1514] Processing DeleteTablet for tablet a06b39cf4f20490cbc68021c17de46b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:37:28 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:48502
I20260806 06:37:28.071162 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 14047
I20260806 06:37:28.080456 14671 tablet_replica.cc:330] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8: stopping tablet replica
I20260806 06:37:28.080610 14671 raft_consensus.cc:2229] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:37:28.080747 14671 raft_consensus.cc:2258] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:37:28.082055 14671 ts_tablet_manager.cc:1905] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:37:28.083717 14670 tablet_replica.cc:330] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250: stopping tablet replica
I20260806 06:37:28.083879 14670 raft_consensus.cc:2229] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:37:28.083959 14670 raft_consensus.cc:2258] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:37:28.084290 14671 ts_tablet_manager.cc:1918] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:37:28.084359 14671 log.cc:1196] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/a06b39cf4f20490cbc68021c17de46b7
I20260806 06:37:28.084679 14671 ts_tablet_manager.cc:1939] T a06b39cf4f20490cbc68021c17de46b7 P 9977c7e74bd9469d8a700ac890de45a8: Deleting consensus metadata
I20260806 06:37:28.085155 13922 catalog_manager.cc:4841] TS 9977c7e74bd9469d8a700ac890de45a8 (127.29.142.130:34125): tablet a06b39cf4f20490cbc68021c17de46b7 (table test-table [id=58e2a388e5684f858d5274ffe9f66fbd]) successfully deleted
I20260806 06:37:28.087246 14670 ts_tablet_manager.cc:1905] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:37:28.089016 14670 ts_tablet_manager.cc:1918] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:37:28.089095 14670 log.cc:1196] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/a06b39cf4f20490cbc68021c17de46b7
I20260806 06:37:28.089807 14670 ts_tablet_manager.cc:1939] T a06b39cf4f20490cbc68021c17de46b7 P 98a6ba7cb9ad41b2b33ef465a0d44250: Deleting consensus metadata
I20260806 06:37:28.094767 13918 catalog_manager.cc:4841] TS 98a6ba7cb9ad41b2b33ef465a0d44250 (127.29.142.131:44065): tablet a06b39cf4f20490cbc68021c17de46b7 (table test-table [id=58e2a388e5684f858d5274ffe9f66fbd]) successfully deleted
W20260806 06:37:28.104278 13922 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:41767 (error 108)
I20260806 06:37:28.104432 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 14233
I20260806 06:37:28.115017 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:37:28.114848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:41767 (local address 127.0.0.1:50902)
0806 06:37:28.114910 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 06:37:28.114966 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:41767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 06:37:28.115372 13922 catalog_manager.cc:4584] TS 4be78fe3632548d791c101a614ac3f7b (127.29.142.129:41767): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a06b39cf4f20490cbc68021c17de46b7: Network error: Client connection negotiation failed: client connection to 127.29.142.129:41767: connect: Connection refused (error 111)
I20260806 06:37:28.122898 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 14421
Aug 06 06:37:28 dist-test-slave-3kk6 krb5kdc[13298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998247, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:37:28.141778 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 13898
I20260806 06:37:28.155037 30266 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 06:37:28.500183 30266 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 06:37:28.500330 30266 mini_ranger.cc:66] Stopping Ranger...
I20260806 06:37:28.855770 30266 mini_ranger.cc:68] Stopped Ranger
2026-08-06 06:37:28.856 UTC [13337] LOG: received smart shutdown request
2026-08-06 06:37:28.860 UTC [13337] LOG: background worker "logical replication launcher" (PID 13351) exited with exit code 1
2026-08-06 06:37:28.860 UTC [13343] LOG: shutting down
2026-08-06 06:37:29.038 UTC [13337] LOG: database system is shut down
2026-08-06T06:37:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50883 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:37:29 dist-test-slave-3kk6 krb5kdc[14686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:37:29 dist-test-slave-3kk6 krb5kdc[14686](info): set up 2 sockets
Aug 06 06:37:29 dist-test-slave-3kk6 krb5kdc[14686](info): commencing operation
krb5kdc: starting...
W20260806 06:37:30.389956 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:37:30 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:37:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:37:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.142.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.142.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/rangeradmin_127.29.142.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/rangeradmin_127.29.142.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.142.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.142.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/rangerlookup_127.29.142.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/rangerlookup_127.29.142.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.142.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.142.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/HTTP_127.29.142.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/HTTP_127.29.142.148@KRBTEST.COM.keytab.
I20260806 06:37:30.578483 30266 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskrotugL/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/postgres -l logfile start
2026-08-06 06:37:33.162 UTC [14727] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 06:37:33.162 UTC [14727] LOG: listening on IPv4 address "127.29.142.148", port 41009
2026-08-06 06:37:33.166 UTC [14727] LOG: listening on Unix socket "/tmp/.s.PGSQL.41009"
2026-08-06 06:37:33.188 UTC [14731] LOG: database system was shut down at 2026-08-06 06:37:31 UTC
2026-08-06 06:37:33.193 UTC [14727] LOG: database system is ready to accept connections
I20260806 06:37:34.327484 30266 mini_postgres.cc:96] Postgres bound to 41009
2026-08-06 06:37:34.334 UTC [14743] FATAL: database "slave" does not exist
127.29.142.148:41009 - accepting connections
I20260806 06:37:34.335135 30266 mini_ranger.cc:161] Starting Ranger...
I20260806 06:37:34.351766 30266 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 06:37:34.829236 30266 mini_ranger.cc:87] Created ranger Postgres database
I20260806 06:37:34.829336 30266 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-admin
2026-08-06 06:37:35,129 [I] DB FLAVOR :POSTGRES
2026-08-06 06:37:35,129 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:37:35,129 [I] Checking connection..
2026-08-06 06:37:35,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 06:37:35,406 [I] Checking connection passed.
2026-08-06 06:37:35,407 [I] --------- Verifying version history table ---------
2026-08-06 06:37:35,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:37:35,670 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:37:35,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:37:35,930 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:37:35,931 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 06:37:35,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 06:37:36.175 UTC [14835] WARNING: there is no transaction in progress
2026-08-06 06:37:36,182 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 06:37:36,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:37:36,444 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 06:37:36,444 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:37:36,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 06:37:36,702 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 06:37:36,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-3kk6.c.gcp-upstream.internal','N') ;"
2026-08-06 06:37:37,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:37:37,452 [I] Table x_portal_user does not exist in database ranger
2026-08-06 06:37:37,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:37:37,718 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 06:37:37,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:37:37,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:37:38,238 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 06:37:38,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 06:37:39.239 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.255 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.272 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.289 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.311 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.330 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.398 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.405 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.420 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.431 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.442 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.457 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.467 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.474 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.481 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:39.488 UTC [14981] WARNING: there is no transaction in progress
2026-08-06 06:37:40,156 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 06:37:40,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 06:37:40,396 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 06:37:40,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:37:40,665 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 06:37:40,666 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:37:40,924 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 06:37:40,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:37:41,194 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:37:41,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-3kk6.c.gcp-upstream.internal';"
2026-08-06 06:37:41,740 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 06:37:41,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 06:37:41,991 [I] DB_PATCHES have already been applied
I20260806 06:37:41.997852 30266 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-admin:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskrotugL/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 06:37:41.997931 30266 mini_ranger.cc:193] Using host: 127.29.142.148
I20260806 06:37:42.001749 30266 mini_ranger.cc:227] Ranger admin URL: http://127.29.142.148:36867
Aug 06, 2026 6:37:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:37:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:37:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:37:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:37:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:37:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:37:42 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 06:37:43.295027 30266 mini_ranger.cc:160] Time spent starting Ranger: real 8.960s user 0.000s sys 0.010s
Aug 06, 2026 6:38:03 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:38:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 6:38:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:38:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:38:03 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:38:04 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 06:38:05.351285 30266 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.142.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.142.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/rangerkms_127.29.142.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.142.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/rangerkms_127.29.142.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.142.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.142.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.142.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/HTTP_127.29.142.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.142.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/HTTP_127.29.142.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 06:38:05 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998285, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 06:38:05.439049 30266 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 06:38:05.449220 30266 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 06:38:06.303683 30266 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 06:38:06.303784 30266 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-kms
I20260806 06:38:06.303799 30266 mini_ranger_kms.cc:227] Using postgres at 127.29.142.148:41009
2026-08-06 06:38:06,511 [I] DB FLAVOR :POSTGRES
2026-08-06 06:38:06,512 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:38:06,512 [I] Checking connection
2026-08-06 06:38:06,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 06:38:06,769 [I] connection success
2026-08-06 06:38:06,770 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 06:38:06,770 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 06:38:06,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 06:38:07,022 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 06:38:07,023 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:38:07,023 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 06:38:07,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.142.148:41009/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 06:38:07,312 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 06:38:07.799775 30266 mini_ranger_kms.cc:310] Created kms service
I20260806 06:38:07.991840 30266 mini_ranger_kms.cc:326] Created kudu user
I20260806 06:38:08.043531 30266 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 06:38:08 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998285, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:38:08.472132 30266 mini_ranger_kms.cc:384] Added ranger policy
I20260806 06:38:08.472293 30266 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-kms:/tmp/dist-test-taskrotugL/build/release/bin/postgresql.jar:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskrotugL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 06:38:08.472316 30266 mini_ranger_kms.cc:240] Using host: 127.29.142.148
I20260806 06:38:08.476433 30266 mini_ranger_kms.cc:276] Ranger KMS PID: 15221
I20260806 06:38:08.476533 30266 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.142.148:56593
Aug 06, 2026 6:38:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:38:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:38:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:38:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:38:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:38:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 06:38:10 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998290, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.142.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:11.154783 30266 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.716s user 0.021s sys 0.168s
I20260806 06:38:11.154897 30266 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 06:38:11.154947 30266 mini_ranger_kms.cc:401] 127.29.142.148:56593/kms/v1/keys
Aug 06 06:38:11 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998290, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.142.148@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
Aug 06 06:38:11 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998290, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.142.148@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
Aug 06 06:38:14 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:38:14.129599 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:46437
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:46437
--ranger_config_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.142.148:56593/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:14.216907 15285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:14.217142 15285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:14.217167 15285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:14.218326 15285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:38:14.218367 15285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:14.218382 15285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:14.218394 15285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:38:14.218407 15285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:38:14.219987 15285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.142.148:56593/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:46437
--ranger_config_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:46437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:14.220196 15285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:14.220384 15285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:14.225942 15292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:14.225898 15294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:14.225999 15285 server_base.cc:1034] running on GCE node
W20260806 06:38:14.225922 15291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:14.226485 15285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:14.226855 15285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:14.228029 15285 hybrid_clock.cc:648] HybridClock initialized: now 1785998294228024 us; error 40 us; skew 500 ppm
Aug 06 06:38:14 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998294, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:14.236654 15285 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:38:14.237188 15285 webserver.cc:460] Webserver started at http://127.29.142.190:41459/ using document root <none> and password file <none>
I20260806 06:38:14.237439 15285 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:14.237500 15285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:14.237679 15285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:38:14 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998294, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:38:14.285408 15285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "2ee8e10bbb514977a4902127395f6223"
format_stamp: "Formatted at 2026-08-06 06:38:14 on dist-test-slave-3kk6"
server_key: "0572bbb1efbd859e1259d63ccf5b5d30"
server_key_iv: "ba293638a493e5454239d9a6c7fe5fa1"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:14.285951 15285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "2ee8e10bbb514977a4902127395f6223"
format_stamp: "Formatted at 2026-08-06 06:38:14 on dist-test-slave-3kk6"
server_key: "0572bbb1efbd859e1259d63ccf5b5d30"
server_key_iv: "ba293638a493e5454239d9a6c7fe5fa1"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:14.287806 15285 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 06:38:14.301654 15307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:14.302004 15285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:38:14.302265 15285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "2ee8e10bbb514977a4902127395f6223"
format_stamp: "Formatted at 2026-08-06 06:38:14 on dist-test-slave-3kk6"
server_key: "0572bbb1efbd859e1259d63ccf5b5d30"
server_key_iv: "ba293638a493e5454239d9a6c7fe5fa1"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:14.302513 15285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:14.332710 15285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:14.334511 15285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:14.334733 15285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:14.339366 15285 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:46437
I20260806 06:38:14.339418 15380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:46437 every 8 connection(s)
I20260806 06:38:14.339802 15285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:38:14.340488 15382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:14.342993 15382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223: Bootstrap starting.
I20260806 06:38:14.343624 15382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:14.343946 15382 log.cc:824] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:14.344695 15382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223: No bootstrap required, opened a new log
I20260806 06:38:14.347224 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 15285
I20260806 06:38:14.347340 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:38:14.347330 15382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee8e10bbb514977a4902127395f6223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46437 } }
I20260806 06:38:14.347541 30266 external_mini_cluster.cc:1428] Setting key 2f58919bc597afb43873fc16e571771a
I20260806 06:38:14.347543 15382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:14.347584 15382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ee8e10bbb514977a4902127395f6223, State: Initialized, Role: FOLLOWER
I20260806 06:38:14.347775 15382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee8e10bbb514977a4902127395f6223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46437 } }
I20260806 06:38:14.347853 15382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:38:14.347877 15382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:38:14.347947 15382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:14.348784 15382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee8e10bbb514977a4902127395f6223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46437 } }
I20260806 06:38:14.348882 15382 leader_election.cc:302] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ee8e10bbb514977a4902127395f6223; no voters:
I20260806 06:38:14.349151 15382 leader_election.cc:288] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:38:14.349189 15394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:38:14.349306 15394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 1 LEADER]: Becoming Leader. State: Replica: 2ee8e10bbb514977a4902127395f6223, State: Running, Role: LEADER
I20260806 06:38:14.349519 15382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:38:14.349490 15394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee8e10bbb514977a4902127395f6223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46437 } }
I20260806 06:38:14.349980 15398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ee8e10bbb514977a4902127395f6223" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee8e10bbb514977a4902127395f6223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46437 } } }
I20260806 06:38:14.350188 15398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:14.350842 15410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:38:14.350904 15397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ee8e10bbb514977a4902127395f6223. Latest consensus state: current_term: 1 leader_uuid: "2ee8e10bbb514977a4902127395f6223" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee8e10bbb514977a4902127395f6223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46437 } } }
I20260806 06:38:14.351065 15397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:14.351227 15382 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:38:14.351423 15410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:38:14.357360 15410 catalog_manager.cc:1347] Generated new cluster ID: 40831612f0274c64a5bc5a513349a5a7
I20260806 06:38:14.357515 15410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:38:14 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:14.364218 15410 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:38:14.364266 15392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:14.348427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37690 (local address 127.29.142.190:46437)
0806 06:38:14.348629 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:14.348638 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:14.348666 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:38:14.348783 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:14.348788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:14.348833 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:14.348996 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:14.349810 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:14.351347 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:14.353420 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:14.355404 (+ 1984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:14.355950 (+ 546us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:14.358877 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:14.358883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:14.358883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:14.358897 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:14.361986 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:14.362450 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:14.362454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:14.362455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:14.362575 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:14.362820 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:14.362821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:14.362822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:14.363158 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:14.363327 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:14.363883 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 06:38:14.364025 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":99,"threads_started":1}
I20260806 06:38:14.365433 15410 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:38:14.370882 15410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223: Generated new TSK 0
I20260806 06:38:14.371106 15410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:38:15.071429 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.142.148:56593/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:46437
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:15.185675 15445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:15.185902 15445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:15.185927 15445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:15.187155 15445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:15.187199 15445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:15.187227 15445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:38:15.188896 15445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.142.148:56593/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:46437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:15.189143 15445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:15.189414 15445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:15.196112 15450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:15.196142 15451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:15.196410 15453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:15.197149 15445 server_base.cc:1034] running on GCE node
I20260806 06:38:15.197381 15445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:15.197647 15445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:15.198827 15445 hybrid_clock.cc:648] HybridClock initialized: now 1785998295198811 us; error 37 us; skew 500 ppm
Aug 06 06:38:15 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998295, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:15.202266 15445 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:38:15.202869 15445 webserver.cc:460] Webserver started at http://127.29.142.129:38457/ using document root <none> and password file <none>
I20260806 06:38:15.203150 15445 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:15.203222 15445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:15.203400 15445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:38:15 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998295, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:38:15.220571 15445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "e545243037fd4da5a48e2e168a9947c5"
format_stamp: "Formatted at 2026-08-06 06:38:15 on dist-test-slave-3kk6"
server_key: "b3693e91e0fc4f5bf7587e617081ae68"
server_key_iv: "192f1b1f9c96ac98e79291d7959be8cb"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:15.221216 15445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "e545243037fd4da5a48e2e168a9947c5"
format_stamp: "Formatted at 2026-08-06 06:38:15 on dist-test-slave-3kk6"
server_key: "b3693e91e0fc4f5bf7587e617081ae68"
server_key_iv: "192f1b1f9c96ac98e79291d7959be8cb"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:15.231029 15445 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.000s
I20260806 06:38:15.249085 15466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:15.249366 15445 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:15.249480 15445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "e545243037fd4da5a48e2e168a9947c5"
format_stamp: "Formatted at 2026-08-06 06:38:15 on dist-test-slave-3kk6"
server_key: "b3693e91e0fc4f5bf7587e617081ae68"
server_key_iv: "192f1b1f9c96ac98e79291d7959be8cb"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:15.249589 15445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:15.263566 15445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:15.266055 15445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:15.266238 15445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:15.266569 15445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:15.267066 15445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:15.267127 15445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:15.267186 15445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:15.267231 15445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:15.274241 15445 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:42457
I20260806 06:38:15.275010 15445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:38:15.279181 15624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:42457 every 8 connection(s)
I20260806 06:38:15.282985 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 15445
I20260806 06:38:15.283092 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:38:15.283279 30266 external_mini_cluster.cc:1428] Setting key 994314bbcad66571dd72544b5aab8442
Aug 06 06:38:15 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998295, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:15.291884 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:15.280054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:46437 (local address 127.29.142.129:56183)
0806 06:38:15.280581 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:15.280634 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:15.281305 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:15.281761 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:15.281766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:15.282050 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:15.282348 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:15.282354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:15.283466 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:15.283467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:15.284736 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:15.284740 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:15.285303 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:15.285304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:15.285456 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:15.286153 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:15.286165 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:15.287941 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:15.290147 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:15.290155 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:15.290158 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:15.290464 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:15.290722 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:15.290723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:15.290723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:15.290872 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:15.291163 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:15.291170 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:15.291340 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 06:38:15.291431 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 06:38:15.291599 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":95,"threads_started":1}
I20260806 06:38:15.292730 15632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:15.280384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:56183 (local address 127.29.142.190:46437)
0806 06:38:15.281528 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:15.281530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:15.281539 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:38:15.281570 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:15.281570 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:15.281584 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:15.282451 (+ 867us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:15.282499 (+ 48us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:15.283335 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:15.284863 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:15.285189 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:15.288046 (+ 2857us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:15.288106 (+ 60us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:15.288107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:15.288107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:15.288112 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:15.290036 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:15.290555 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:15.290556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:15.290557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:15.290620 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:15.290950 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:15.290950 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:15.290951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:15.291090 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:15.292500 (+ 1410us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:15.292553 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 06:38:15.292650 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1108,"thread_start_us":88,"threads_started":1}
I20260806 06:38:15.293164 15627 heartbeater.cc:347] Connected to a master server at 127.29.142.190:46437
I20260806 06:38:15.293282 15627 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:15.293514 15627 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
I20260806 06:38:15.294140 15333 ts_manager.cc:194] Registered new tserver with Master: e545243037fd4da5a48e2e168a9947c5 (127.29.142.129:42457)
I20260806 06:38:15.295095 15333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:56183
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:38:15.338439 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.142.148:56593/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:46437
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:38:15 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998295, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 06:38:15.475430 15636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:15.475816 15636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:15.475900 15636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:15.477759 15636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:15.478008 15636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:15.478111 15636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:38:15.480479 15636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.142.148:56593/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:46437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:15.481006 15636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:15.481360 15636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:15.488536 15648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:15.488783 15651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:15.488518 15649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:15.489962 15636 server_base.cc:1034] running on GCE node
I20260806 06:38:15.490202 15636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:15.490479 15636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:15.505985 15636 hybrid_clock.cc:648] HybridClock initialized: now 1785998295504500 us; error 1505 us; skew 500 ppm
Aug 06 06:38:15 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998295, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:15.509743 15636 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:38:15.510381 15636 webserver.cc:460] Webserver started at http://127.29.142.130:40919/ using document root <none> and password file <none>
I20260806 06:38:15.510702 15636 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:15.510844 15636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:15.511054 15636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:38:15 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998295, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:38:15.525274 15636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "17863b791cba4a35a1b188b82a941b75"
format_stamp: "Formatted at 2026-08-06 06:38:15 on dist-test-slave-3kk6"
server_key: "b48881a86939c0b474137f5cec81f917"
server_key_iv: "29656ef2fef3189f037d4d773e8d6933"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:15.525825 15636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "17863b791cba4a35a1b188b82a941b75"
format_stamp: "Formatted at 2026-08-06 06:38:15 on dist-test-slave-3kk6"
server_key: "b48881a86939c0b474137f5cec81f917"
server_key_iv: "29656ef2fef3189f037d4d773e8d6933"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:15.529623 15636 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260806 06:38:15.553120 15660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:15.553395 15636 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260806 06:38:15.553498 15636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "17863b791cba4a35a1b188b82a941b75"
format_stamp: "Formatted at 2026-08-06 06:38:15 on dist-test-slave-3kk6"
server_key: "b48881a86939c0b474137f5cec81f917"
server_key_iv: "29656ef2fef3189f037d4d773e8d6933"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:15.553606 15636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:15.577327 15636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:15.580096 15636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:15.580329 15636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:15.580619 15636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:15.581069 15636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:15.581159 15636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:15.581244 15636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:15.581305 15636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:15.589022 15636 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:33555
I20260806 06:38:15.589319 15815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:33555 every 8 connection(s)
I20260806 06:38:15.589627 15636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:38:15.593580 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 15636
I20260806 06:38:15.593695 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:38:15.593878 30266 external_mini_cluster.cc:1428] Setting key 9ea2ab824313ea9e5e395576c6abd33d
Aug 06 06:38:15 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998295, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:15.606480 15632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:15.590514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:55535 (local address 127.29.142.190:46437)
0806 06:38:15.590569 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:15.590572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:15.590906 (+ 334us) server_negotiation.cc:407] Connection header received
0806 06:38:15.592500 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:15.592502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:15.592522 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:15.592591 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:15.593227 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:15.594029 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:15.597086 (+ 3057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:15.597322 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:15.597425 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:15.599756 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:15.599758 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:15.599758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:15.599774 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:15.604157 (+ 4383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:15.604737 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:15.604738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:15.604739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:15.604808 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:15.605167 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:15.605168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:15.605169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:15.605332 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:15.605388 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:15.606266 (+ 878us) server_negotiation.cc:299] Negotiation successful
0806 06:38:15.606373 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:38:15.607370 15822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:15.590512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:46437 (local address 127.29.142.130:55535)
0806 06:38:15.590752 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:15.590799 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:15.592365 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:15.592696 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:15.592699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:15.592902 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:15.593108 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:15.593113 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:15.594119 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:15.594120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:15.596954 (+ 2834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:15.596958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:15.597456 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:15.597457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:15.597581 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:15.598205 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:15.598214 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:15.599637 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:15.604288 (+ 4651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:15.604298 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:15.604301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:15.604637 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:15.604882 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:15.604882 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:15.604883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:15.605086 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:15.605425 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:15.605432 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:15.605649 (+ 217us) client_negotiation.cc:769] Sending connection context
0806 06:38:15.606915 (+ 1266us) client_negotiation.cc:240] Negotiation successful
0806 06:38:15.607104 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":115,"threads_started":1}
I20260806 06:38:15.609064 15816 heartbeater.cc:347] Connected to a master server at 127.29.142.190:46437
I20260806 06:38:15.609247 15816 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:15.609500 15816 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
I20260806 06:38:15.609956 15333 ts_manager.cc:194] Registered new tserver with Master: 17863b791cba4a35a1b188b82a941b75 (127.29.142.130:33555)
I20260806 06:38:15.610456 15333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:55535
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:38:15.644259 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.142.148:56593/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:46437
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:15.786163 15828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:15.786500 15828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:15.786545 15828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:15.787842 15828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:15.787922 15828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:15.787971 15828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:38:15.789470 15828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.142.148:56593/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:46437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:15.789768 15828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:15.790010 15828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:15.795367 15833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:15.795531 15836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:15.795827 15834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:15.796835 15828 server_base.cc:1034] running on GCE node
I20260806 06:38:15.797060 15828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:15.797333 15828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:15.798493 15828 hybrid_clock.cc:648] HybridClock initialized: now 1785998295798473 us; error 44 us; skew 500 ppm
Aug 06 06:38:15 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998295, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:15.801220 15828 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:38:15.801688 15828 webserver.cc:460] Webserver started at http://127.29.142.131:37149/ using document root <none> and password file <none>
I20260806 06:38:15.801946 15828 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:15.802019 15828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:15.802187 15828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:38:15 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998295, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:38:15.813278 15828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "b1d12d5905324458846fec0c38fccd2a"
format_stamp: "Formatted at 2026-08-06 06:38:15 on dist-test-slave-3kk6"
server_key: "653d7a12b507c567f4a026fff3a4d6ef"
server_key_iv: "667ac689b22ee09ffe42dbb8aab189b8"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:15.813894 15828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "b1d12d5905324458846fec0c38fccd2a"
format_stamp: "Formatted at 2026-08-06 06:38:15 on dist-test-slave-3kk6"
server_key: "653d7a12b507c567f4a026fff3a4d6ef"
server_key_iv: "667ac689b22ee09ffe42dbb8aab189b8"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:15.817976 15828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260806 06:38:15.843504 15843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:15.849025 15828 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260806 06:38:15.849292 15828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "b1d12d5905324458846fec0c38fccd2a"
format_stamp: "Formatted at 2026-08-06 06:38:15 on dist-test-slave-3kk6"
server_key: "653d7a12b507c567f4a026fff3a4d6ef"
server_key_iv: "667ac689b22ee09ffe42dbb8aab189b8"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:15.849532 15828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:15.895716 15828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:15.898442 15828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:15.898720 15828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:15.899070 15828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:15.899554 15828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:15.899660 15828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:15.899756 15828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:15.899821 15828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:15.908134 15828 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:35589
I20260806 06:38:15.908702 15828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:38:15.909960 15993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:35589 every 8 connection(s)
I20260806 06:38:15.911397 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 15828
I20260806 06:38:15.911496 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:38:15.911741 30266 external_mini_cluster.cc:1428] Setting key 4f1750389f2def4dde8a0cd5d98efcc5
Aug 06 06:38:15 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998295, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:15.925248 15632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:15.911076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:58697 (local address 127.29.142.190:46437)
0806 06:38:15.911137 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:15.911140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:15.911545 (+ 405us) server_negotiation.cc:407] Connection header received
0806 06:38:15.913124 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:15.913125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:15.913140 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:15.913174 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:15.914141 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:15.915027 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:15.916662 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:15.916997 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:15.917046 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:15.920025 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:15.920027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:15.920028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:15.920036 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:15.922952 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:15.924094 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:15.924095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:15.924096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:15.924161 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:15.924452 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:15.924453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:15.924453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:15.924595 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:15.924639 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:15.925054 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 06:38:15.925146 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:38:15.925860 15996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:15.910695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:46437 (local address 127.29.142.131:58697)
0806 06:38:15.911374 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:15.911436 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:15.912983 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:15.913388 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:15.913393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:15.913689 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:15.914013 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:15.914021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:15.915157 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:15.915158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:15.916540 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:15.916544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:15.917162 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:15.917162 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:15.917315 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:15.918027 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:15.918038 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:15.919904 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:15.923637 (+ 3733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:15.923645 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:15.923648 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:15.923993 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:15.924237 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:15.924237 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:15.924238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:15.924377 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:15.924684 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:15.924690 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:15.924885 (+ 195us) client_negotiation.cc:769] Sending connection context
0806 06:38:15.925403 (+ 518us) client_negotiation.cc:240] Negotiation successful
0806 06:38:15.925598 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":113,"threads_started":1}
I20260806 06:38:15.926316 15994 heartbeater.cc:347] Connected to a master server at 127.29.142.190:46437
I20260806 06:38:15.926422 15994 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:15.926633 15994 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
I20260806 06:38:15.927027 15333 ts_manager.cc:194] Registered new tserver with Master: b1d12d5905324458846fec0c38fccd2a (127.29.142.131:35589)
I20260806 06:38:15.927537 15333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:58697
I20260806 06:38:15.935307 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:38:15.936162 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 15445
I20260806 06:38:16.046129 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 15636
I20260806 06:38:16.057976 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 15828
Aug 06 06:38:16 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998295, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:38:16.069638 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 15285
I20260806 06:38:16.083046 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:46437
--webserver_interface=127.29.142.190
--webserver_port=41459
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:46437
--ranger_config_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.142.148:56593/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.142.148:56593/kms with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:16.172672 15998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:16.172948 15998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:16.172991 15998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:16.174153 15998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:38:16.174230 15998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:16.174255 15998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:16.174278 15998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:38:16.174307 15998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:38:16.176079 15998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.142.148:56593/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:46437
--ranger_config_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:46437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=41459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:16.176366 15998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:16.176604 15998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:16.183156 16009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:16.183166 16007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:16.183182 16011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:16.183473 15998 server_base.cc:1034] running on GCE node
I20260806 06:38:16.183768 15998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:16.184109 15998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:16.185276 15998 hybrid_clock.cc:648] HybridClock initialized: now 1785998296185253 us; error 44 us; skew 500 ppm
Aug 06 06:38:16 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998296, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:16.188205 15998 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:38:16.188727 15998 webserver.cc:460] Webserver started at http://127.29.142.190:41459/ using document root <none> and password file <none>
I20260806 06:38:16.188982 15998 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:16.189050 15998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:38:16 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998296, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:38:16.234433 15998 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:16.235484 16021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:16.235714 15998 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:38:16.235833 15998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "2ee8e10bbb514977a4902127395f6223"
format_stamp: "Formatted at 2026-08-06 06:38:14 on dist-test-slave-3kk6"
server_key: "0572bbb1efbd859e1259d63ccf5b5d30"
server_key_iv: "ba293638a493e5454239d9a6c7fe5fa1"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:16.236378 15998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:16.264902 15998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:16.267436 15998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:16.267602 15998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:16.273156 15998 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:46437
I20260806 06:38:16.273319 16089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:46437 every 8 connection(s)
I20260806 06:38:16.273703 15998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:38:16.275175 16092 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:38:16.275982 16092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223: Bootstrap starting.
I20260806 06:38:16.281193 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 15998
I20260806 06:38:16.281504 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.142.148:56593/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:42457
--local_ip_for_outbound_sockets=127.29.142.129
--tserver_master_addrs=127.29.142.190:46437
--webserver_port=38457
--webserver_interface=127.29.142.129
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.142.148:56593/kms with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:38:16.285322 16092 log.cc:824] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:16.287406 16092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:38:16.287770 16092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223: Bootstrap complete.
I20260806 06:38:16.290340 16092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee8e10bbb514977a4902127395f6223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46437 } }
I20260806 06:38:16.290614 16092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ee8e10bbb514977a4902127395f6223, State: Initialized, Role: FOLLOWER
I20260806 06:38:16.290856 16092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee8e10bbb514977a4902127395f6223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46437 } }
I20260806 06:38:16.290952 16092 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:38:16.291006 16092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:38:16.291054 16092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:38:16.292142 16092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee8e10bbb514977a4902127395f6223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46437 } }
I20260806 06:38:16.292277 16092 leader_election.cc:302] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ee8e10bbb514977a4902127395f6223; no voters:
I20260806 06:38:16.292487 16092 leader_election.cc:288] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:38:16.292591 16098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:38:16.292781 16092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:38:16.292769 16098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [term 2 LEADER]: Becoming Leader. State: Replica: 2ee8e10bbb514977a4902127395f6223, State: Running, Role: LEADER
I20260806 06:38:16.294221 16098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee8e10bbb514977a4902127395f6223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46437 } }
I20260806 06:38:16.294555 16098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ee8e10bbb514977a4902127395f6223. Latest consensus state: current_term: 2 leader_uuid: "2ee8e10bbb514977a4902127395f6223" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee8e10bbb514977a4902127395f6223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46437 } } }
I20260806 06:38:16.294628 16098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:16.294791 16099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2ee8e10bbb514977a4902127395f6223" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ee8e10bbb514977a4902127395f6223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46437 } } }
I20260806 06:38:16.294857 16099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:16.295435 16092 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:38:16.298050 16114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:38:16.299063 16114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:38:16.299314 16114 catalog_manager.cc:1259] Loaded cluster ID: 40831612f0274c64a5bc5a513349a5a7
I20260806 06:38:16.299365 16114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:38:16.301491 16114 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:38:16.301770 16114 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223: Loaded TSK: 0
I20260806 06:38:16.302067 16114 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:38:16.405067 16094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:16.405371 16094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:16.405411 16094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:16.407140 16094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:16.407209 16094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:16.407254 16094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:38:16.409411 16094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.142.148:56593/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:42457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=38457
--tserver_master_addrs=127.29.142.190:46437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:16.409673 16094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:16.409961 16094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:16.416824 16136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:16.416885 16137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:16.416824 16139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:16.417413 16094 server_base.cc:1034] running on GCE node
I20260806 06:38:16.417725 16094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:16.418016 16094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:16.421303 16094 hybrid_clock.cc:648] HybridClock initialized: now 1785998296421259 us; error 56 us; skew 500 ppm
Aug 06 06:38:16 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998296, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:16.427425 16094 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:38:16.428090 16094 webserver.cc:460] Webserver started at http://127.29.142.129:38457/ using document root <none> and password file <none>
I20260806 06:38:16.428406 16094 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:16.428515 16094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:38:16 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998296, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:38:16.453979 16094 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260806 06:38:16.455340 16153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:16.455519 16094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:38:16.455605 16094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "e545243037fd4da5a48e2e168a9947c5"
format_stamp: "Formatted at 2026-08-06 06:38:15 on dist-test-slave-3kk6"
server_key: "b3693e91e0fc4f5bf7587e617081ae68"
server_key_iv: "192f1b1f9c96ac98e79291d7959be8cb"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:16.456118 16094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:16.517457 16094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:16.520833 16094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:16.521104 16094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:16.521466 16094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:16.521930 16094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:16.521993 16094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:16.522037 16094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:16.522068 16094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:16.530695 16094 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:42457
I20260806 06:38:16.531281 16094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:38:16.533488 16313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:42457 every 8 connection(s)
I20260806 06:38:16.542912 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 16094
I20260806 06:38:16.543247 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.142.148:56593/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:33555
--local_ip_for_outbound_sockets=127.29.142.130
--tserver_master_addrs=127.29.142.190:46437
--webserver_port=40919
--webserver_interface=127.29.142.130
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.142.148:56593/kms with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:38:16 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998296, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:16.566347 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:16.533881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:46437 (local address 127.29.142.129:58595)
0806 06:38:16.537401 (+ 3520us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:16.538456 (+ 1055us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:16.539205 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:16.539939 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:16.539943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:16.540197 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:16.540506 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:16.540513 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.542779 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.542781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:16.545089 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:16.545093 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.546883 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.546885 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:16.547023 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:16.556812 (+ 9789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:16.556828 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:16.558863 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:16.564108 (+ 5245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:16.564120 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:16.564123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:16.564486 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:16.564845 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:16.564846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:16.564846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:16.565000 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:16.565576 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:16.565583 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:16.565749 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 06:38:16.565871 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 06:38:16.566066 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3468,"thread_start_us":92,"threads_started":1}
I20260806 06:38:16.567217 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:16.533957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:58595 (local address 127.29.142.190:46437)
0806 06:38:16.534205 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:16.534216 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:16.538628 (+ 4412us) server_negotiation.cc:407] Connection header received
0806 06:38:16.539542 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:16.539549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:16.539611 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:16.539774 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:16.540652 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.542610 (+ 1958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.545211 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.546595 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.546745 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:16.559077 (+ 12332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:16.559084 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:16.559085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:16.559102 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:16.563963 (+ 4861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:16.564613 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:16.564617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:16.564618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:16.564751 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:16.565083 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:16.565084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:16.565085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:16.565448 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:16.565531 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:16.566865 (+ 1334us) server_negotiation.cc:299] Negotiation successful
0806 06:38:16.567007 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":87,"threads_started":1}
I20260806 06:38:16.568451 16314 heartbeater.cc:347] Connected to a master server at 127.29.142.190:46437
I20260806 06:38:16.568555 16314 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:16.568746 16314 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
W20260806 06:38:16.570779 16314 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:46437 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:38:16.570861 16314 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:16.570931 16314 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
I20260806 06:38:16.571395 16314 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:16.571473 16314 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
W20260806 06:38:16.571858 16314 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:38:16.662478 16318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:16.662720 16318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:16.662746 16318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:16.663889 16318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:16.663931 16318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:16.663957 16318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:38:16.665514 16318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.142.148:56593/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:33555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=40919
--tserver_master_addrs=127.29.142.190:46437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:16.665743 16318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:16.665998 16318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:16.673548 16327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:16.673573 16329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:16.673637 16325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:16.673856 16318 server_base.cc:1034] running on GCE node
I20260806 06:38:16.674072 16318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:16.674381 16318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:16.675567 16318 hybrid_clock.cc:648] HybridClock initialized: now 1785998296675534 us; error 53 us; skew 500 ppm
Aug 06 06:38:16 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998296, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:16.679016 16318 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:38:16.679625 16318 webserver.cc:460] Webserver started at http://127.29.142.130:40919/ using document root <none> and password file <none>
I20260806 06:38:16.679942 16318 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:16.680037 16318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:38:16 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998296, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:38:16.703253 16318 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.001s
I20260806 06:38:16.704355 16340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:16.706001 16318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 06:38:16.706125 16318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "17863b791cba4a35a1b188b82a941b75"
format_stamp: "Formatted at 2026-08-06 06:38:15 on dist-test-slave-3kk6"
server_key: "b48881a86939c0b474137f5cec81f917"
server_key_iv: "29656ef2fef3189f037d4d773e8d6933"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:16.706606 16318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:16.724712 16318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:16.730286 16318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:16.730564 16318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:16.730970 16318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:16.731501 16318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:16.731602 16318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:16.731709 16318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:16.731779 16318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:16.740432 16318 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:33555
I20260806 06:38:16.741032 16318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:38:16.746284 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 16318
I20260806 06:38:16.746595 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.142.148:56593/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:35589
--local_ip_for_outbound_sockets=127.29.142.131
--tserver_master_addrs=127.29.142.190:46437
--webserver_port=37149
--webserver_interface=127.29.142.131
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.142.148:56593/kms with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:38:16.748857 16494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:33555 every 8 connection(s)
Aug 06 06:38:16 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998296, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:16.774156 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:16.751399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:52009 (local address 127.29.142.190:46437)
0806 06:38:16.751454 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:16.751456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:16.754976 (+ 3520us) server_negotiation.cc:407] Connection header received
0806 06:38:16.755717 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:16.755719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:16.755738 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:16.755786 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:16.756683 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.757485 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.764055 (+ 6570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.764389 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.764439 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:16.769498 (+ 5059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:16.769500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:16.769501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:16.769510 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:16.771386 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:16.771958 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:16.771959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:16.771960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:16.772024 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:16.772310 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:16.772311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:16.772311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:16.772466 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:16.772521 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:16.773946 (+ 1425us) server_negotiation.cc:299] Negotiation successful
0806 06:38:16.774048 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:38:16.774727 16499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:16.751417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:46437 (local address 127.29.142.130:52009)
0806 06:38:16.754796 (+ 3379us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:16.754854 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:16.755572 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:16.756000 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:16.756005 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:16.756259 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:16.756558 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:16.756565 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.757612 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.757613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:16.762517 (+ 4904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:16.762520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.764831 (+ 2311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.764832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:16.764999 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:16.765984 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:16.765995 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:16.769361 (+ 3366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:16.771500 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:16.771508 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:16.771511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:16.771860 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:16.772092 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:16.772093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:16.772093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:16.772232 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:16.772559 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:16.772566 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:16.772757 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 06:38:16.774272 (+ 1515us) client_negotiation.cc:240] Negotiation successful
0806 06:38:16.774458 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3323,"thread_start_us":97,"threads_started":1}
I20260806 06:38:16.775311 16495 heartbeater.cc:347] Connected to a master server at 127.29.142.190:46437
I20260806 06:38:16.775426 16495 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:16.775641 16495 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
W20260806 06:38:16.776389 16495 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:46437 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:38:16.776466 16495 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:16.776535 16495 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
I20260806 06:38:16.776917 16495 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:16.776994 16495 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
W20260806 06:38:16.777359 16495 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:38:16.863605 16498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:16.863821 16498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:16.863845 16498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:16.865084 16498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:16.865127 16498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:16.865154 16498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:38:16.866755 16498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.142.148:56593/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:35589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=37149
--tserver_master_addrs=127.29.142.190:46437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:16.866995 16498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:16.867254 16498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:16.872716 16509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:16.872736 16511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:16.873059 16508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:16.878162 16498 server_base.cc:1034] running on GCE node
I20260806 06:38:16.878437 16498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:16.878821 16498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:16.879981 16498 hybrid_clock.cc:648] HybridClock initialized: now 1785998296879953 us; error 48 us; skew 500 ppm
Aug 06 06:38:16 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998296, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:16.882985 16498 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:38:16.883531 16498 webserver.cc:460] Webserver started at http://127.29.142.131:37149/ using document root <none> and password file <none>
I20260806 06:38:16.883791 16498 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:16.883898 16498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:38:16 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998296, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:38:16.907168 16498 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260806 06:38:16.908401 16521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:16.908591 16498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:16.908708 16498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "b1d12d5905324458846fec0c38fccd2a"
format_stamp: "Formatted at 2026-08-06 06:38:15 on dist-test-slave-3kk6"
server_key: "653d7a12b507c567f4a026fff3a4d6ef"
server_key_iv: "667ac689b22ee09ffe42dbb8aab189b8"
server_key_version: "kuduclusterkey@0"
I20260806 06:38:16.909193 16498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:16.930771 16498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:16.933708 16498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:16.933888 16498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:16.934150 16498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:16.934506 16498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:16.934546 16498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:16.934583 16498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:16.934608 16498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:16.942531 16498 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:35589
I20260806 06:38:16.942770 16680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:35589 every 8 connection(s)
I20260806 06:38:16.943096 16498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:38:16.945849 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 16498
Aug 06 06:38:16 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998296, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:16.956002 16687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:16.944021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:46437 (local address 127.29.142.131:37335)
0806 06:38:16.944231 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:16.944281 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:16.944917 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:16.945216 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:16.945221 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:16.945465 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:16.945780 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:16.945788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.947766 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.947767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:16.949071 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:16.949075 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.949687 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.949689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:16.949854 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:16.950516 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:16.950527 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:16.952232 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:16.954306 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:16.954313 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:16.954316 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:16.954610 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:16.954894 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:16.954895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:16.954895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:16.955029 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:16.955319 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:16.955325 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:16.955487 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 06:38:16.955593 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 06:38:16.955758 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
I20260806 06:38:16.956571 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:16.944338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:37335 (local address 127.29.142.190:46437)
0806 06:38:16.944385 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:16.944387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:16.944401 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:38:16.945056 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:16.945058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:16.945071 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:16.945143 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:16.945903 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.946771 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.949183 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.949489 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.949572 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:16.952374 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:16.952375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:16.952376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:16.952383 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:16.954207 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:16.954759 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:16.954760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:16.954761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:16.954824 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:16.955102 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:16.955103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:16.955103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:16.955230 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:16.955291 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:16.956392 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0806 06:38:16.956476 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:38:16.956909 16683 heartbeater.cc:347] Connected to a master server at 127.29.142.190:46437
I20260806 06:38:16.957015 16683 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:16.957214 16683 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
W20260806 06:38:16.957947 16683 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:46437 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:38:16.958019 16683 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:16.958083 16683 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
I20260806 06:38:16.958396 16683 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:16.958472 16683 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
W20260806 06:38:16.958817 16683 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:38:16.959281 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:16.946431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37694 (local address 127.29.142.190:46437)
0806 06:38:16.949430 (+ 2999us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:16.949432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:16.949440 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:38:16.949462 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:16.949463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:16.949474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:16.949533 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:16.950448 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.952018 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.953472 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:16.953798 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:16.953862 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:16.955407 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:16.955409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:16.955409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:16.955417 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:16.957483 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:16.957885 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:16.957886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:16.957886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:16.957944 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:16.958210 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:16.958211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:16.958211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:16.958335 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:16.959047 (+ 712us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:16.959091 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 06:38:16.959176 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2973,"thread_start_us":60,"threads_started":1}
Aug 06 06:38:17 dist-test-slave-3kk6 krb5kdc[14686](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998297, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:17.572053 16314 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:17.572229 16314 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
I20260806 06:38:17.572635 16044 ts_manager.cc:194] Registered new tserver with Master: e545243037fd4da5a48e2e168a9947c5 (127.29.142.129:42457)
I20260806 06:38:17.573690 16044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:58595
Aug 06 06:38:17 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998297, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:38:17.777570 16495 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:17.777737 16495 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
I20260806 06:38:17.778126 16044 ts_manager.cc:194] Registered new tserver with Master: 17863b791cba4a35a1b188b82a941b75 (127.29.142.130:33555)
I20260806 06:38:17.778708 16044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:52009
I20260806 06:38:17.958994 16683 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:17.959167 16683 heartbeater.cc:511] Master 127.29.142.190:46437 requested a full tablet report, sending...
I20260806 06:38:17.959484 16044 ts_manager.cc:194] Registered new tserver with Master: b1d12d5905324458846fec0c38fccd2a (127.29.142.131:35589)
I20260806 06:38:17.959978 16044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:37335
I20260806 06:38:17.967393 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:38:17.978756 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:17.968637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37704 (local address 127.29.142.190:46437)
0806 06:38:17.968750 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:17.968753 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:17.970854 (+ 2101us) server_negotiation.cc:407] Connection header received
0806 06:38:17.970959 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:17.970960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:17.970973 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:17.971016 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:17.971789 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.972581 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.973597 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.973886 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.973925 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:17.975583 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:17.975584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:17.975585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:17.975593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:17.977447 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:17.977842 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:17.977843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:17.977843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:17.977912 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:17.978134 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:17.978135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:17.978135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:17.978280 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:17.978388 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:17.978531 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 06:38:17.978628 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":64,"threads_started":1}
I20260806 06:38:17.981112 16044 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:38:17.983863 16044 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:38:17.993119 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:17.987727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35154 (local address 127.29.142.130:33555)
0806 06:38:17.987885 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:17.987890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:17.988549 (+ 659us) server_negotiation.cc:407] Connection header received
0806 06:38:17.988604 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:17.988607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:17.988625 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:17.988715 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:17.988984 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.990095 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.991730 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.992583 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.992640 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:17.992701 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:17.992983 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 06:38:17.993027 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260806 06:38:17.993515 16729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:17.987428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42457 (local address 127.0.0.1:34562)
0806 06:38:17.988707 (+ 1279us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:17.988716 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:17.988767 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:17.989059 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:17.989060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:17.989062 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:17.989108 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:17.989110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.990599 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.990601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:17.991786 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:17.991790 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.993150 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.993152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:17.993230 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:17.993234 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:38:17.993286 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:38:17.993402 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1265,"thread_start_us":105,"threads_started":1}
I20260806 06:38:17.993641 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:17.987423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:33555 (local address 127.0.0.1:35154)
0806 06:38:17.988415 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:17.988457 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:17.988531 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:17.988825 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:17.988828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:17.988831 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:17.988898 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:17.988900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.990198 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.990199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:17.991577 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:17.991582 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.992780 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.992781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:17.992863 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:17.992877 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:38:17.993521 (+ 644us) client_negotiation.cc:240] Negotiation successful
0806 06:38:17.993570 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":962,"thread_start_us":103,"threads_started":1}
I20260806 06:38:17.994150 16726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:17.987714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.29.142.129:42457)
0806 06:38:17.988659 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:17.988664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:17.988825 (+ 161us) server_negotiation.cc:407] Connection header received
0806 06:38:17.988863 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:17.988865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:17.988881 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:17.988973 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:17.989183 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.990496 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.991890 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.992983 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.993058 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:17.993112 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:17.994022 (+ 910us) server_negotiation.cc:299] Negotiation successful
0806 06:38:17.994066 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":913,"thread_start_us":61,"threads_started":1}
I20260806 06:38:17.995065 16227 tablet_service.cc:1467] Processing CreateTablet for tablet 467c052889404a06bb2af1b1473c8d4c (DEFAULT_TABLE table=test-table [id=092868cd47624b8db250ac5a264c0211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:38:17.995540 16227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467c052889404a06bb2af1b1473c8d4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:17.996223 16402 tablet_service.cc:1467] Processing CreateTablet for tablet 467c052889404a06bb2af1b1473c8d4c (DEFAULT_TABLE table=test-table [id=092868cd47624b8db250ac5a264c0211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:38:17.996567 16402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467c052889404a06bb2af1b1473c8d4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:17.997838 16729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:17.987935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:35589 (local address 127.0.0.1:32894)
0806 06:38:17.993898 (+ 5963us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:17.993908 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:17.993946 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:17.994294 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:17.994295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:17.994297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:17.994329 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:17.994330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.995573 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.995574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:17.996567 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:17.996570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.997602 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.997603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:17.997662 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:17.997665 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:38:17.997718 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:38:17.997759 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5951,"thread_start_us":70,"threads_started":1}
I20260806 06:38:17.998119 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:17.988034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32894 (local address 127.29.142.131:35589)
0806 06:38:17.988181 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:17.988186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:17.994010 (+ 5824us) server_negotiation.cc:407] Connection header received
0806 06:38:17.994062 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:17.994064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:17.994082 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:17.994181 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:17.994402 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.995467 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.996676 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:17.997425 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:17.997505 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:17.997563 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:17.997995 (+ 432us) server_negotiation.cc:299] Negotiation successful
0806 06:38:17.998041 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260806 06:38:17.998806 16586 tablet_service.cc:1467] Processing CreateTablet for tablet 467c052889404a06bb2af1b1473c8d4c (DEFAULT_TABLE table=test-table [id=092868cd47624b8db250ac5a264c0211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:38:17.999143 16586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467c052889404a06bb2af1b1473c8d4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:17.999351 16732 tablet_bootstrap.cc:492] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5: Bootstrap starting.
I20260806 06:38:18.000036 16732 tablet_bootstrap.cc:654] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:18.000456 16733 tablet_bootstrap.cc:492] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75: Bootstrap starting.
I20260806 06:38:18.000478 16732 log.cc:824] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:18.001205 16733 tablet_bootstrap.cc:654] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:18.001320 16732 tablet_bootstrap.cc:492] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5: No bootstrap required, opened a new log
I20260806 06:38:18.001406 16732 ts_tablet_manager.cc:1397] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:38:18.001590 16733 log.cc:824] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:18.002344 16733 tablet_bootstrap.cc:492] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75: No bootstrap required, opened a new log
I20260806 06:38:18.002409 16733 ts_tablet_manager.cc:1397] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:38:18.003705 16732 raft_consensus.cc:348] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d12d5905324458846fec0c38fccd2a" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35589 } } peers { permanent_uuid: "17863b791cba4a35a1b188b82a941b75" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 33555 } } peers { permanent_uuid: "e545243037fd4da5a48e2e168a9947c5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42457 } }
I20260806 06:38:18.003880 16732 raft_consensus.cc:374] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:18.003935 16732 raft_consensus.cc:729] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e545243037fd4da5a48e2e168a9947c5, State: Initialized, Role: FOLLOWER
I20260806 06:38:18.004940 16732 consensus_queue.cc:260] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d12d5905324458846fec0c38fccd2a" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35589 } } peers { permanent_uuid: "17863b791cba4a35a1b188b82a941b75" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 33555 } } peers { permanent_uuid: "e545243037fd4da5a48e2e168a9947c5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42457 } }
I20260806 06:38:18.005000 16733 raft_consensus.cc:348] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d12d5905324458846fec0c38fccd2a" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35589 } } peers { permanent_uuid: "17863b791cba4a35a1b188b82a941b75" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 33555 } } peers { permanent_uuid: "e545243037fd4da5a48e2e168a9947c5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42457 } }
I20260806 06:38:18.005157 16733 raft_consensus.cc:374] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:18.005190 16733 raft_consensus.cc:729] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17863b791cba4a35a1b188b82a941b75, State: Initialized, Role: FOLLOWER
I20260806 06:38:18.005364 16732 ts_tablet_manager.cc:1428] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 06:38:18.005399 16733 consensus_queue.cc:260] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d12d5905324458846fec0c38fccd2a" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35589 } } peers { permanent_uuid: "17863b791cba4a35a1b188b82a941b75" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 33555 } } peers { permanent_uuid: "e545243037fd4da5a48e2e168a9947c5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42457 } }
I20260806 06:38:18.005568 16314 heartbeater.cc:503] Master 127.29.142.190:46437 was elected leader, sending a full tablet report...
I20260806 06:38:18.005713 16733 ts_tablet_manager.cc:1428] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:38:18.005813 16495 heartbeater.cc:503] Master 127.29.142.190:46437 was elected leader, sending a full tablet report...
I20260806 06:38:18.010900 16739 tablet_bootstrap.cc:492] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a: Bootstrap starting.
I20260806 06:38:18.011854 16739 tablet_bootstrap.cc:654] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:18.012229 16739 log.cc:824] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:18.013034 16739 tablet_bootstrap.cc:492] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a: No bootstrap required, opened a new log
I20260806 06:38:18.013106 16739 ts_tablet_manager.cc:1397] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:38:18.015772 16739 raft_consensus.cc:348] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d12d5905324458846fec0c38fccd2a" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35589 } } peers { permanent_uuid: "17863b791cba4a35a1b188b82a941b75" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 33555 } } peers { permanent_uuid: "e545243037fd4da5a48e2e168a9947c5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42457 } }
I20260806 06:38:18.015949 16739 raft_consensus.cc:374] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:18.015981 16739 raft_consensus.cc:729] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d12d5905324458846fec0c38fccd2a, State: Initialized, Role: FOLLOWER
I20260806 06:38:18.016151 16739 consensus_queue.cc:260] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d12d5905324458846fec0c38fccd2a" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35589 } } peers { permanent_uuid: "17863b791cba4a35a1b188b82a941b75" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 33555 } } peers { permanent_uuid: "e545243037fd4da5a48e2e168a9947c5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42457 } }
I20260806 06:38:18.016472 16739 ts_tablet_manager.cc:1428] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:38:18.016535 16683 heartbeater.cc:503] Master 127.29.142.190:46437 was elected leader, sending a full tablet report...
W20260806 06:38:18.034174 16315 tablet.cc:2369] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 06:38:18 dist-test-slave-3kk6 krb5kdc[14686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998297, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for HTTP/127.29.142.148@KRBTEST.COM
I20260806 06:38:18.059823 16741 raft_consensus.cc:482] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:38:18.059971 16741 raft_consensus.cc:504] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d12d5905324458846fec0c38fccd2a" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35589 } } peers { permanent_uuid: "17863b791cba4a35a1b188b82a941b75" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 33555 } } peers { permanent_uuid: "e545243037fd4da5a48e2e168a9947c5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42457 } }
I20260806 06:38:18.060410 16741 leader_election.cc:288] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1d12d5905324458846fec0c38fccd2a (127.29.142.131:35589), e545243037fd4da5a48e2e168a9947c5 (127.29.142.129:42457)
I20260806 06:38:18.065609 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:18.061148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:34191 (local address 127.29.142.131:35589)
0806 06:38:18.061194 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:18.061196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:18.061701 (+ 505us) server_negotiation.cc:407] Connection header received
0806 06:38:18.061811 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:18.061812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:18.061823 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:18.061854 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:18.062078 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:18.062841 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:18.064638 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:18.065173 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:18.065224 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:18.065235 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:18.065485 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 06:38:18.065534 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:38:18.066109 16745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:18.060679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:35589 (local address 127.29.142.130:34191)
0806 06:38:18.061628 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:18.061638 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:18.061732 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:18.061924 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:18.061925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:18.061927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:18.062006 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:18.062008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:18.063323 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:18.063324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:18.064528 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:18.064530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:18.065332 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:18.065333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:18.065399 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:18.065402 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:38:18.065930 (+ 528us) client_negotiation.cc:240] Negotiation successful
0806 06:38:18.065977 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"thread_start_us":131,"threads_started":1}
I20260806 06:38:18.066763 16613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467c052889404a06bb2af1b1473c8d4c" candidate_uuid: "17863b791cba4a35a1b188b82a941b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d12d5905324458846fec0c38fccd2a" is_pre_election: true
I20260806 06:38:18.066859 16726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:18.061391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:54427 (local address 127.29.142.129:42457)
0806 06:38:18.061484 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:18.061486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:18.063510 (+ 2024us) server_negotiation.cc:407] Connection header received
0806 06:38:18.063623 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:18.063624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:18.063636 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:18.063667 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:18.063899 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:18.064576 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:18.065898 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:18.066447 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:18.066497 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:18.066504 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:18.066749 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 06:38:18.066793 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 06:38:18.066951 16613 raft_consensus.cc:2454] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17863b791cba4a35a1b188b82a941b75 in term 0.
I20260806 06:38:18.067098 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:18.060944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42457 (local address 127.29.142.130:54427)
0806 06:38:18.063431 (+ 2487us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:18.063445 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:18.063542 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:18.063740 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:18.063741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:18.063743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:18.063823 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:18.063825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:18.064688 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:18.064689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:18.065789 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:18.065792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:18.066568 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:18.066569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:18.066641 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:18.066644 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:38:18.066981 (+ 337us) client_negotiation.cc:240] Negotiation successful
0806 06:38:18.067022 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2459,"thread_start_us":71,"threads_started":1}
I20260806 06:38:18.067327 16344 leader_election.cc:302] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17863b791cba4a35a1b188b82a941b75, b1d12d5905324458846fec0c38fccd2a; no voters:
I20260806 06:38:18.067476 16741 raft_consensus.cc:2781] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:38:18.067528 16741 raft_consensus.cc:482] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:38:18.067561 16741 raft_consensus.cc:3037] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:18.067667 16251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467c052889404a06bb2af1b1473c8d4c" candidate_uuid: "17863b791cba4a35a1b188b82a941b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e545243037fd4da5a48e2e168a9947c5" is_pre_election: true
I20260806 06:38:18.067845 16251 raft_consensus.cc:2454] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17863b791cba4a35a1b188b82a941b75 in term 0.
I20260806 06:38:18.068643 16741 raft_consensus.cc:504] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d12d5905324458846fec0c38fccd2a" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35589 } } peers { permanent_uuid: "17863b791cba4a35a1b188b82a941b75" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 33555 } } peers { permanent_uuid: "e545243037fd4da5a48e2e168a9947c5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42457 } }
I20260806 06:38:18.068802 16741 leader_election.cc:288] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [CANDIDATE]: Term 1 election: Requested vote from peers b1d12d5905324458846fec0c38fccd2a (127.29.142.131:35589), e545243037fd4da5a48e2e168a9947c5 (127.29.142.129:42457)
I20260806 06:38:18.069209 16251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467c052889404a06bb2af1b1473c8d4c" candidate_uuid: "17863b791cba4a35a1b188b82a941b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e545243037fd4da5a48e2e168a9947c5"
I20260806 06:38:18.069296 16251 raft_consensus.cc:3037] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:18.070444 16251 raft_consensus.cc:2454] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17863b791cba4a35a1b188b82a941b75 in term 1.
I20260806 06:38:18.070669 16343 leader_election.cc:302] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17863b791cba4a35a1b188b82a941b75, e545243037fd4da5a48e2e168a9947c5; no voters:
I20260806 06:38:18.070863 16613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467c052889404a06bb2af1b1473c8d4c" candidate_uuid: "17863b791cba4a35a1b188b82a941b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1d12d5905324458846fec0c38fccd2a"
I20260806 06:38:18.070958 16613 raft_consensus.cc:3037] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:18.071859 16613 raft_consensus.cc:2454] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17863b791cba4a35a1b188b82a941b75 in term 1.
I20260806 06:38:18.072186 16741 raft_consensus.cc:2781] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:38:18.072342 16741 raft_consensus.cc:686] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 1 LEADER]: Becoming Leader. State: Replica: 17863b791cba4a35a1b188b82a941b75, State: Running, Role: LEADER
I20260806 06:38:18.072434 16741 consensus_queue.cc:237] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d12d5905324458846fec0c38fccd2a" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35589 } } peers { permanent_uuid: "17863b791cba4a35a1b188b82a941b75" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 33555 } } peers { permanent_uuid: "e545243037fd4da5a48e2e168a9947c5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42457 } }
I20260806 06:38:18.073417 16044 catalog_manager.cc:5495] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17863b791cba4a35a1b188b82a941b75 (127.29.142.130). New cstate: current_term: 1 leader_uuid: "17863b791cba4a35a1b188b82a941b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d12d5905324458846fec0c38fccd2a" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17863b791cba4a35a1b188b82a941b75" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 33555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e545243037fd4da5a48e2e168a9947c5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42457 } health_report { overall_health: UNKNOWN } } }
I20260806 06:38:18.087692 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:18.084068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46750 (local address 127.29.142.130:33555)
0806 06:38:18.084118 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:18.084121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:18.084132 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:38:18.084222 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:18.084223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:18.084246 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:18.084328 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:38:18.084526 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:18.085242 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:18.086410 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:18.086731 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:18.086796 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:18.087023 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:38:18.087162 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:38:18.087242 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:18.087583 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 06:38:18.087609 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:38:18.090093 16251 raft_consensus.cc:1261] T 467c052889404a06bb2af1b1473c8d4c P e545243037fd4da5a48e2e168a9947c5 [term 1 FOLLOWER]: Refusing update from remote peer 17863b791cba4a35a1b188b82a941b75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:38:18.090461 16613 raft_consensus.cc:1261] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a [term 1 FOLLOWER]: Refusing update from remote peer 17863b791cba4a35a1b188b82a941b75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:38:18.091329 16741 consensus_queue.cc:1035] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e545243037fd4da5a48e2e168a9947c5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:38:18.091509 16741 consensus_queue.cc:1035] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1d12d5905324458846fec0c38fccd2a" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:38:18.105259 16753 mvcc.cc:205] Tried to move back new op lower bound from 7315449028973314048 to 7315449028905279488. Current Snapshot: MvccSnapshot[applied={T|T < 7315449028973314048}]
I20260806 06:38:18.107455 16757 mvcc.cc:205] Tried to move back new op lower bound from 7315449028973314048 to 7315449028905279488. Current Snapshot: MvccSnapshot[applied={T|T < 7315449028973314048}]
I20260806 06:38:18.111088 16044 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:38:18.111202 16044 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:38:18.112353 16044 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ee8e10bbb514977a4902127395f6223: Sending DeleteTablet for 3 replicas of tablet 467c052889404a06bb2af1b1473c8d4c
I20260806 06:38:18.112798 16227 tablet_service.cc:1514] Processing DeleteTablet for tablet 467c052889404a06bb2af1b1473c8d4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:18 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:34562
I20260806 06:38:18.113122 16402 tablet_service.cc:1514] Processing DeleteTablet for tablet 467c052889404a06bb2af1b1473c8d4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:18 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:35154
I20260806 06:38:18.113569 16586 tablet_service.cc:1514] Processing DeleteTablet for tablet 467c052889404a06bb2af1b1473c8d4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:18 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:32894
I20260806 06:38:18.114249 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 16094
I20260806 06:38:18.121625 16769 tablet_replica.cc:330] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75: stopping tablet replica
I20260806 06:38:18.121846 16769 raft_consensus.cc:2229] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:38:18.122078 16769 raft_consensus.cc:2258] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:38:18.123127 16769 ts_tablet_manager.cc:1905] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:38:18.124966 16769 ts_tablet_manager.cc:1918] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:38:18.125125 16769 log.cc:1196] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/467c052889404a06bb2af1b1473c8d4c
I20260806 06:38:18.125487 16769 ts_tablet_manager.cc:1939] T 467c052889404a06bb2af1b1473c8d4c P 17863b791cba4a35a1b188b82a941b75: Deleting consensus metadata
I20260806 06:38:18.126091 16025 catalog_manager.cc:4841] TS 17863b791cba4a35a1b188b82a941b75 (127.29.142.130:33555): tablet 467c052889404a06bb2af1b1473c8d4c (table test-table [id=092868cd47624b8db250ac5a264c0211]) successfully deleted
I20260806 06:38:18.126471 16770 tablet_replica.cc:330] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a: stopping tablet replica
I20260806 06:38:18.126785 16770 raft_consensus.cc:2229] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:38:18.127920 16770 raft_consensus.cc:2258] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:38:18.128527 16770 ts_tablet_manager.cc:1905] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:38:18.130249 16770 ts_tablet_manager.cc:1918] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:38:18.134752 16770 log.cc:1196] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/467c052889404a06bb2af1b1473c8d4c
I20260806 06:38:18.135161 16770 ts_tablet_manager.cc:1939] T 467c052889404a06bb2af1b1473c8d4c P b1d12d5905324458846fec0c38fccd2a: Deleting consensus metadata
I20260806 06:38:18.135725 16024 catalog_manager.cc:4841] TS b1d12d5905324458846fec0c38fccd2a (127.29.142.131:35589): tablet 467c052889404a06bb2af1b1473c8d4c (table test-table [id=092868cd47624b8db250ac5a264c0211]) successfully deleted
W20260806 06:38:18.136103 16023 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:42457 (error 108)
I20260806 06:38:18.136904 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 16318
I20260806 06:38:18.148226 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:18.147474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42457 (local address 127.0.0.1:42940)
0806 06:38:18.148108 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:18.148167 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:42457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607}
W20260806 06:38:18.149096 16023 catalog_manager.cc:4584] TS e545243037fd4da5a48e2e168a9947c5 (127.29.142.129:42457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 467c052889404a06bb2af1b1473c8d4c: Network error: Client connection negotiation failed: client connection to 127.29.142.129:42457: connect: Connection refused (error 111)
I20260806 06:38:18.154160 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 16498
I20260806 06:38:18.167619 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 15998
I20260806 06:38:18.177866 30266 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 06:38:18.532081 30266 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 06:38:18.532204 30266 mini_ranger.cc:66] Stopping Ranger...
I20260806 06:38:18.886865 30266 mini_ranger.cc:68] Stopped Ranger
2026-08-06 06:38:18.887 UTC [14727] LOG: received smart shutdown request
2026-08-06 06:38:18.891 UTC [14727] LOG: background worker "logical replication launcher" (PID 14741) exited with exit code 1
2026-08-06 06:38:18.892 UTC [14732] LOG: shutting down
2026-08-06 06:38:19.059 UTC [14727] LOG: database system is shut down
2026-08-06T06:38:19Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50020 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:38:19 dist-test-slave-3kk6 krb5kdc[16783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:38:19 dist-test-slave-3kk6 krb5kdc[16783](info): set up 2 sockets
Aug 06 06:38:19 dist-test-slave-3kk6 krb5kdc[16783](info): commencing operation
krb5kdc: starting...
W20260806 06:38:20.404574 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:38:20 dist-test-slave-3kk6 krb5kdc[16783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:38:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:38:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:38:20.538218 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:39541
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:39541,127.29.142.189:43715,127.29.142.188:40241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:20.623812 16806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:20.624048 16806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:20.624073 16806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:20.625254 16806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:38:20.625303 16806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:20.625317 16806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:20.625330 16806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:38:20.625344 16806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:38:20.626919 16806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:39541,127.29.142.189:43715,127.29.142.188:40241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:39541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:20.627180 16806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:20.627435 16806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:20.632680 16816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:20.632691 16813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:20.632812 16806 server_base.cc:1034] running on GCE node
W20260806 06:38:20.632699 16812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:20.633203 16806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:20.633577 16806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:20.634721 16806 hybrid_clock.cc:648] HybridClock initialized: now 1785998300634705 us; error 48 us; skew 500 ppm
Aug 06 06:38:20 dist-test-slave-3kk6 krb5kdc[16783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:20.640276 16806 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:38:20.640769 16806 webserver.cc:460] Webserver started at http://127.29.142.190:42637/ using document root <none> and password file <none>
I20260806 06:38:20.640965 16806 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:20.641008 16806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:20.641099 16806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:38:20.642158 16806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "813bbedb07e14129940c690e0553d223"
format_stamp: "Formatted at 2026-08-06 06:38:20 on dist-test-slave-3kk6"
server_key: "e2e64d01e4b3f48a967822d05f1ee8e2"
server_key_iv: "ba572f78bbbdbea7ec50b7f148891553"
server_key_version: "encryptionkey@0"
I20260806 06:38:20.642493 16806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "813bbedb07e14129940c690e0553d223"
format_stamp: "Formatted at 2026-08-06 06:38:20 on dist-test-slave-3kk6"
server_key: "e2e64d01e4b3f48a967822d05f1ee8e2"
server_key_iv: "ba572f78bbbdbea7ec50b7f148891553"
server_key_version: "encryptionkey@0"
I20260806 06:38:20.643985 16806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:38:20.644781 16826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:20.645033 16806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:20.645143 16806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "813bbedb07e14129940c690e0553d223"
format_stamp: "Formatted at 2026-08-06 06:38:20 on dist-test-slave-3kk6"
server_key: "e2e64d01e4b3f48a967822d05f1ee8e2"
server_key_iv: "ba572f78bbbdbea7ec50b7f148891553"
server_key_version: "encryptionkey@0"
I20260806 06:38:20.645229 16806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:20.659478 16806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:20.661501 16806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:20.661659 16806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:20.665930 16806 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:39541
I20260806 06:38:20.666014 16897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:39541 every 8 connection(s)
I20260806 06:38:20.666368 16806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:38:20.667318 16898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:20.668926 16898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:38:20.674341 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 16806
I20260806 06:38:20.674472 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:38:20.674748 30266 external_mini_cluster.cc:1428] Setting key c8cc672bce99dea0bc5208fa7534c2c8
Aug 06 06:38:20 dist-test-slave-3kk6 krb5kdc[16783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:20.680351 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.669721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39541 (local address 127.0.0.1:37122)
0806 06:38:20.669906 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.669950 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:20.670526 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:20.670936 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:20.670939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:20.671207 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:20.671369 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:20.671373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.672415 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.672416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:20.673137 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:20.673139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.673671 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.673673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:20.673753 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:20.674368 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:20.674379 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:20.676274 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:20.678757 (+ 2483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:20.678764 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:20.678766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:20.678972 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:20.679284 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:20.679285 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:20.679286 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:20.679401 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:20.679886 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:20.679892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:20.679955 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 06:38:20.680031 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 06:38:20.680170 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
I20260806 06:38:20.680481 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.669876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37122 (local address 127.29.142.190:39541)
0806 06:38:20.670034 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:20.670038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:20.670069 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:38:20.670797 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:20.670799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:20.670818 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:20.670906 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:20.671464 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.672317 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.673230 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.673580 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.673687 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:20.676391 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:20.676401 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:20.676402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:20.676412 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:20.678588 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:20.679069 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:20.679073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:20.679073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:20.679192 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:20.679472 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:20.679473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:20.679474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:20.679810 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:20.679907 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:20.680092 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 06:38:20.680286 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260806 06:38:20.681242 16898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:38:20.681624 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.681512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:43715 (local address 127.0.0.1:57596)
0806 06:38:20.681559 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.681604 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:43715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:38:20.681785 16827 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.142.189:43715: connect: Connection refused (error 111)
I20260806 06:38:20.682189 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.682125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:43715 (local address 127.0.0.1:57602)
0806 06:38:20.682149 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.682173 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:43715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260806 06:38:20.682365 16898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.189:43715: Network error: Client connection negotiation failed: client connection to 127.29.142.189:43715: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.29.142.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.189.keytab.
Entry for principal kudu/127.29.142.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.189.keytab.
I20260806 06:38:20.702484 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.189:43715
--webserver_interface=127.29.142.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.189.keytab
--principal=kudu/127.29.142.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:39541,127.29.142.189:43715,127.29.142.188:40241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:38:20.731616 16898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } attempt: 1
I20260806 06:38:20.732316 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.732152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:43715 (local address 127.0.0.1:57604)
0806 06:38:20.732234 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.732285 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:43715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 06:38:20.732736 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.732621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:43715 (local address 127.0.0.1:57608)
0806 06:38:20.732676 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.732710 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:43715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 06:38:20.732842 16898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.189:43715: Network error: Client connection negotiation failed: client connection to 127.29.142.189:43715: connect: Connection refused (error 111)
W20260806 06:38:20.787947 16915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:20.788236 16915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:20.788271 16915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:20.789424 16915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:38:20.789497 16915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:20.789521 16915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:20.789542 16915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:38:20.789575 16915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:38:20.791144 16915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:39541,127.29.142.189:43715,127.29.142.188:40241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.189.keytab
--principal=kudu/127.29.142.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.189:43715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.142.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:20.791437 16915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:20.791715 16915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:38:20.796998 16915 server_base.cc:1034] running on GCE node
W20260806 06:38:20.796918 16925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:20.796918 16923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:20.796926 16922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:20.797423 16915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:20.797748 16915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:20.798920 16915 hybrid_clock.cc:648] HybridClock initialized: now 1785998300798906 us; error 30 us; skew 500 ppm
I20260806 06:38:20.801002 16898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } attempt: 2
Aug 06 06:38:20 dist-test-slave-3kk6 krb5kdc[16783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:20.801441 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.801285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:43715 (local address 127.0.0.1:57618)
0806 06:38:20.801356 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.801401 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:43715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 06:38:20.801841 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.801739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:43715 (local address 127.0.0.1:57622)
0806 06:38:20.801789 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.801818 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:43715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:38:20.801882 16915 init.cc:377] Logged in from keytab as kudu/127.29.142.189@KRBTEST.COM (short username kudu)
W20260806 06:38:20.801964 16898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.189:43715: Network error: Client connection negotiation failed: client connection to 127.29.142.189:43715: connect: Connection refused (error 111)
I20260806 06:38:20.802392 16915 webserver.cc:460] Webserver started at http://127.29.142.189:46687/ using document root <none> and password file <none>
I20260806 06:38:20.802628 16915 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:20.802719 16915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:20.802875 16915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:38:20.804028 16915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/data/instance:
uuid: "97bf62a7f7c54edab719a941fc9b19b6"
format_stamp: "Formatted at 2026-08-06 06:38:20 on dist-test-slave-3kk6"
server_key: "1bc524ef70345b5f0afdf55b5e82f537"
server_key_iv: "1beac638b791f51b86842d68a391e5b4"
server_key_version: "encryptionkey@0"
I20260806 06:38:20.804442 16915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/wal/instance:
uuid: "97bf62a7f7c54edab719a941fc9b19b6"
format_stamp: "Formatted at 2026-08-06 06:38:20 on dist-test-slave-3kk6"
server_key: "1bc524ef70345b5f0afdf55b5e82f537"
server_key_iv: "1beac638b791f51b86842d68a391e5b4"
server_key_version: "encryptionkey@0"
I20260806 06:38:20.805986 16915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:38:20.806847 16935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:20.807045 16915 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:38:20.807142 16915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/wal
uuid: "97bf62a7f7c54edab719a941fc9b19b6"
format_stamp: "Formatted at 2026-08-06 06:38:20 on dist-test-slave-3kk6"
server_key: "1bc524ef70345b5f0afdf55b5e82f537"
server_key_iv: "1beac638b791f51b86842d68a391e5b4"
server_key_version: "encryptionkey@0"
I20260806 06:38:20.807232 16915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:20.831259 16915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:20.833817 16915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:20.833994 16915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:20.838279 16915 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.189:43715
I20260806 06:38:20.838614 17006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.189:43715 every 8 connection(s)
I20260806 06:38:20.838740 16915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
I20260806 06:38:20.839485 17008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:20.841109 17008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:38:20 dist-test-slave-3kk6 krb5kdc[16783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.189@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:20.848559 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 16915
I20260806 06:38:20.848696 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-1/wal/instance
I20260806 06:38:20.848918 30266 external_mini_cluster.cc:1428] Setting key 31ef0ec55a1e717520d7df7174a8df1d
I20260806 06:38:20.850620 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.841889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37124 (local address 127.29.142.190:39541)
0806 06:38:20.841957 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:20.841959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:20.842185 (+ 226us) server_negotiation.cc:407] Connection header received
0806 06:38:20.842738 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:20.842739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:20.842756 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:20.842822 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:20.843470 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.843975 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.844979 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.845187 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.845266 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:20.847501 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:20.847502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:20.847502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:20.847510 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:20.849155 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:20.849592 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:20.849593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:20.849593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:20.849665 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:20.849913 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:20.849914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:20.849914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:20.850074 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:20.850145 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:20.850387 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 06:38:20.850486 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:38:20.851028 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.841841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39541 (local address 127.0.0.1:37124)
0806 06:38:20.842056 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.842093 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:20.842590 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:20.842897 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:20.842900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:20.843168 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:20.843367 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:20.843371 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.844080 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.844081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:20.844872 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:20.844874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.845273 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.845275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:20.845399 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:20.845920 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:20.845928 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:20.847378 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:20.849257 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:20.849264 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:20.849267 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:20.849512 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:20.849739 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:20.849739 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:20.849739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:20.849846 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:20.850139 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:20.850143 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:20.850257 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 06:38:20.850341 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:38:20.850461 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":107,"threads_started":1}
I20260806 06:38:20.851701 17008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:38:20 dist-test-slave-3kk6 krb5kdc[16783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.189@KRBTEST.COM for kudu/127.29.142.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.142.188@KRBTEST.COM; defaulting to no policy
I20260806 06:38:20.860176 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.851999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:43715 (local address 127.0.0.1:57638)
0806 06:38:20.852051 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.852063 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:20.852105 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:20.852404 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:20.852405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:20.852524 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:20.852575 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:20.852576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.853929 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.853930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:20.854420 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:20.854421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.855206 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.855207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:20.855281 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:20.855833 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:20.855838 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:20.857055 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:20.859113 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:20.859114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:20.859114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:20.859278 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:20.859512 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:20.859513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:20.859513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:20.859615 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:20.859912 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:20.859913 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:20.859954 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:38:20.860012 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:38:20.860102 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:38:20.860311 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.852058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57638 (local address 127.29.142.189:43715)
0806 06:38:20.852244 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:20.852249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:20.852272 (+ 23us) server_negotiation.cc:407] Connection header received
0806 06:38:20.852295 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:20.852297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:20.852312 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:20.852392 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:20.852649 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.853831 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.854509 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.855082 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.855175 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:20.857152 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:20.857161 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:20.857162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:20.857169 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:20.859032 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:20.859356 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:20.859359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:20.859359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:20.859415 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:20.859681 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:20.859681 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:20.859682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:20.859845 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:20.859912 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:20.860044 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 06:38:20.860164 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":153,"thread_start_us":104,"threads_started":1}
I20260806 06:38:20.861001 17008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.188@KRBTEST.COM" created.
I20260806 06:38:20.861445 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.861300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54830)
0806 06:38:20.861350 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.861421 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 06:38:20.861646 16936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
I20260806 06:38:20.862123 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.862040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54834)
0806 06:38:20.862082 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.862107 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:38:20.862249 17008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:40241: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.188.keytab.
Entry for principal kudu/127.29.142.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.188.keytab.
I20260806 06:38:20.877513 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.188:40241
--webserver_interface=127.29.142.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.188.keytab
--principal=kudu/127.29.142.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:39541,127.29.142.189:43715,127.29.142.188:40241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:38:20.893160 16898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } attempt: 3
Aug 06 06:38:20 dist-test-slave-3kk6 krb5kdc[16783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for kudu/127.29.142.189@KRBTEST.COM
I20260806 06:38:20.900596 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.893610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:43715 (local address 127.0.0.1:57650)
0806 06:38:20.893682 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.893703 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:20.893805 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:20.893991 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:20.893992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:20.894167 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:20.894238 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:20.894241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.894986 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.894987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:20.895508 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:20.895510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.895900 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.895901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:20.895982 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:20.896529 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:20.896534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:20.897810 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:20.899481 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:20.899483 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:20.899483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:20.899669 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:20.899918 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:20.899919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:20.899919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:20.899985 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:20.900252 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:20.900253 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:20.900300 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 06:38:20.900379 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:38:20.900490 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:38:20.900596 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.893697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57650 (local address 127.29.142.189:43715)
0806 06:38:20.893768 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:20.893771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:20.893787 (+ 16us) server_negotiation.cc:407] Connection header received
0806 06:38:20.893890 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:20.893891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:20.893912 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:20.893970 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:20.894338 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.894900 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.895601 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:20.895820 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:20.895903 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:20.897947 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:20.897949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:20.897949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:20.897956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:20.899389 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:20.899780 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:20.899784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:20.899784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:20.899842 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:20.900052 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:20.900053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:20.900053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:20.900169 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:20.900246 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:20.900416 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 06:38:20.900492 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:38:20.901033 16898 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:38:20.901460 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.901332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54844)
0806 06:38:20.901398 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.901436 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 06:38:20.901805 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.901691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54854)
0806 06:38:20.901758 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.901784 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 06:38:20.901902 16898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:40241: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
I20260806 06:38:20.911504 17008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } attempt: 1
I20260806 06:38:20.912086 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.911931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54866)
0806 06:38:20.911998 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.912050 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:38:20.912493 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.912393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54868)
0806 06:38:20.912442 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.912473 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 06:38:20.912597 17008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:40241: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
I20260806 06:38:20.933048 16898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } attempt: 1
I20260806 06:38:20.933645 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.933467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54874)
0806 06:38:20.933545 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.933602 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 06:38:20.934144 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.934037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54876)
0806 06:38:20.934089 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.934121 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:38:20.934255 16898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:40241: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
W20260806 06:38:20.966970 17017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:20.967217 17017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:20.967242 17017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:20.968459 17017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:38:20.968515 17017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:20.968530 17017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:20.968544 17017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:38:20.968556 17017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:38:20.970157 17017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:39541,127.29.142.189:43715,127.29.142.188:40241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.188.keytab
--principal=kudu/127.29.142.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.188:40241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.142.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:20.970469 17017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:20.970860 17017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:20.976042 17023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:20.976080 17025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:20.976058 17022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:20.976361 17017 server_base.cc:1034] running on GCE node
I20260806 06:38:20.976634 17017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:20.976990 17017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:20.978168 17017 hybrid_clock.cc:648] HybridClock initialized: now 1785998300978140 us; error 44 us; skew 500 ppm
Aug 06 06:38:20 dist-test-slave-3kk6 krb5kdc[16783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:20.980729 17008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } attempt: 2
I20260806 06:38:20.981083 17017 init.cc:377] Logged in from keytab as kudu/127.29.142.188@KRBTEST.COM (short username kudu)
I20260806 06:38:20.981155 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.981010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54884)
0806 06:38:20.981074 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.981110 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 06:38:20.981531 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:20.981446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54888)
0806 06:38:20.981480 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:20.981505 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 06:38:20.981657 17017 webserver.cc:460] Webserver started at http://127.29.142.188:45607/ using document root <none> and password file <none>
W20260806 06:38:20.981767 17008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:40241: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
I20260806 06:38:20.981973 17017 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:20.982049 17017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:20.982195 17017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:38:20.983471 17017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/data/instance:
uuid: "9fab03e056174e3aa02e40e1a9c4846d"
format_stamp: "Formatted at 2026-08-06 06:38:20 on dist-test-slave-3kk6"
server_key: "4c92056e3b16363fc1c8097714a07b02"
server_key_iv: "b0d800d4d50e214f62d9dfb5e33c7e18"
server_key_version: "encryptionkey@0"
I20260806 06:38:20.983887 17017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/wal/instance:
uuid: "9fab03e056174e3aa02e40e1a9c4846d"
format_stamp: "Formatted at 2026-08-06 06:38:20 on dist-test-slave-3kk6"
server_key: "4c92056e3b16363fc1c8097714a07b02"
server_key_iv: "b0d800d4d50e214f62d9dfb5e33c7e18"
server_key_version: "encryptionkey@0"
I20260806 06:38:20.985405 17017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:38:20.986311 17037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:20.986518 17017 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:20.986616 17017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/wal
uuid: "9fab03e056174e3aa02e40e1a9c4846d"
format_stamp: "Formatted at 2026-08-06 06:38:20 on dist-test-slave-3kk6"
server_key: "4c92056e3b16363fc1c8097714a07b02"
server_key_iv: "b0d800d4d50e214f62d9dfb5e33c7e18"
server_key_version: "encryptionkey@0"
I20260806 06:38:20.986944 17017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:21.005441 17017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:21.007985 17017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:21.008188 17017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:21.009414 16898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } attempt: 2
I20260806 06:38:21.010015 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.009846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54892)
0806 06:38:21.009931 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:21.009976 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:38:21.010490 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.010350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54898)
0806 06:38:21.010422 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:21.010462 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 06:38:21.010591 16898 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:40241: Network error: Client connection negotiation failed: client connection to 127.29.142.188:40241: connect: Connection refused (error 111)
I20260806 06:38:21.013130 17017 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.188:40241
I20260806 06:38:21.013182 17116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.188:40241 every 8 connection(s)
I20260806 06:38:21.013612 17017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
I20260806 06:38:21.014562 17117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:21.016440 17117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:38:21 dist-test-slave-3kk6 krb5kdc[16783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.188@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:21.023375 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 17017
I20260806 06:38:21.023527 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998137760399-30266-0/minicluster-data/master-2/wal/instance
I20260806 06:38:21.023732 30266 external_mini_cluster.cc:1428] Setting key 66b82f44113c1c15ebe2235d3e8a5128
I20260806 06:38:21.027279 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.017299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37130 (local address 127.29.142.190:39541)
0806 06:38:21.017370 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:21.017372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:21.017572 (+ 200us) server_negotiation.cc:407] Connection header received
0806 06:38:21.018019 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:21.018020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:21.018030 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:21.018100 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:21.018785 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.019263 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.020264 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.020490 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.020560 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.023119 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:21.023120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:21.023120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:21.023129 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:21.025737 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.026172 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.026174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.026174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.026241 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.026496 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.026497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.026497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.026704 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:21.026786 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:21.027091 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 06:38:21.027181 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:38:21.027505 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.017243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:39541 (local address 127.0.0.1:37130)
0806 06:38:21.017426 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:21.017472 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:21.017926 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:21.018156 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:21.018159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:21.018433 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:21.018631 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:21.018635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.019352 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.019353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:21.020150 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:21.020152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.020589 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.020590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:21.020707 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.021338 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:21.021347 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:21.022983 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:21.025831 (+ 2848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:21.025838 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:21.025840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:21.026092 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:21.026316 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:21.026316 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:21.026317 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:21.026430 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:21.026778 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:21.026785 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:21.026943 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 06:38:21.027057 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:38:21.027232 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
I20260806 06:38:21.028133 17117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:38:21 dist-test-slave-3kk6 krb5kdc[16783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.188@KRBTEST.COM
Aug 06 06:38:21 dist-test-slave-3kk6 krb5kdc[16783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.188@KRBTEST.COM for kudu/127.29.142.189@KRBTEST.COM
I20260806 06:38:21.034406 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.024243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54900 (local address 127.29.142.188:40241)
0806 06:38:21.024524 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:21.024530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:21.024557 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:38:21.024586 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:21.024589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:21.024606 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:21.024705 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:21.025508 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.026515 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.027479 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.027974 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.028056 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.030871 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:21.030881 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:21.030882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:21.030889 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:21.032914 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.033370 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.033374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.033374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.033445 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.033655 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.033656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.033657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.033882 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:21.034078 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:21.034137 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 06:38:21.034250 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":116,"threads_started":1}
I20260806 06:38:21.037245 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.028967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57652 (local address 127.29.142.189:43715)
0806 06:38:21.029050 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:21.029052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:21.029059 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:38:21.029080 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:21.029081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:21.029089 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:21.029155 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:21.029458 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.030132 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.030950 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.031232 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.031291 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.033272 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:21.033273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:21.033273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:21.033281 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:21.036039 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.036410 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.036411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.036411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.036482 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.036756 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.036757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.036757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.036865 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:21.036917 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:21.037097 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 06:38:21.037177 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 06:38:21.037283 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.028791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:43715 (local address 127.0.0.1:57652)
0806 06:38:21.028870 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:21.028885 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:21.028933 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:21.029166 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:21.029167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:21.029318 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:21.029372 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:21.029373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.030226 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.030227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:21.030880 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:21.030882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.031324 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.031326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:21.031414 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.031817 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:21.031821 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:21.033169 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:21.036127 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:21.036128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:21.036128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:21.036323 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:21.036561 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:21.036562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:21.036562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:21.036638 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:21.036936 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:21.036937 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:21.036987 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 06:38:21.037069 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:38:21.037177 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 06:38:21.037767 17117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:38:21 dist-test-slave-3kk6 krb5kdc[16783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
Aug 06 06:38:21 dist-test-slave-3kk6 krb5kdc[16783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.188@KRBTEST.COM for kudu/127.29.142.188@KRBTEST.COM
I20260806 06:38:21.045033 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.035556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37140 (local address 127.29.142.190:39541)
0806 06:38:21.035612 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:21.035614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:21.035750 (+ 136us) server_negotiation.cc:407] Connection header received
0806 06:38:21.035776 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:21.035777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:21.035788 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:21.035826 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:21.036448 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.037277 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.038326 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.038593 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.038701 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.041802 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:21.041803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:21.041804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:21.041814 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:21.043651 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.044134 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.044135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.044136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.044200 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.044406 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.044407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.044408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.044569 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:21.044820 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:21.044889 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 06:38:21.044961 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:38:21.045905 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.038107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54912)
0806 06:38:21.038221 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:21.038235 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:21.038288 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:21.038423 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:21.038424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:21.038562 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:21.038607 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:21.038609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.039531 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.039533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:21.040401 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:21.040405 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.040958 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.040959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:21.041001 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.041492 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:21.041498 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:21.042891 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:21.044757 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:21.044758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:21.044759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:21.045004 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:21.045232 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:21.045232 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:21.045233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:21.045290 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:21.045591 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:21.045591 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:21.045631 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 06:38:21.045694 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:38:21.045800 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 06:38:21.046084 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.038171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54912 (local address 127.29.142.188:40241)
0806 06:38:21.038221 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:21.038223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:21.038283 (+ 60us) server_negotiation.cc:407] Connection header received
0806 06:38:21.038355 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:21.038356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:21.038367 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:21.038421 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:21.038777 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.039438 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.040535 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.040868 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.040951 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.042987 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:21.042989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:21.042989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:21.042998 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:21.044663 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.045086 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.045088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.045088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.045153 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.045349 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.045350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.045350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.045505 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:21.045567 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:21.045719 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:38:21.045933 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":32}
I20260806 06:38:21.047744 17117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d: Bootstrap starting.
I20260806 06:38:21.048784 17117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:21.049132 17117 log.cc:824] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:21.049930 17117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d: No bootstrap required, opened a new log
Aug 06 06:38:21 dist-test-slave-3kk6 krb5kdc[16783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.189@KRBTEST.COM
I20260806 06:38:21.052042 17117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } }
I20260806 06:38:21.052184 17117 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:21.052208 17117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fab03e056174e3aa02e40e1a9c4846d, State: Initialized, Role: FOLLOWER
I20260806 06:38:21.052382 17117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } }
I20260806 06:38:21.052757 17117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:38:21.052767 17128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } } }
I20260806 06:38:21.052901 17128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:38:21.054474 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.045844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57668 (local address 127.29.142.189:43715)
0806 06:38:21.045909 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:21.045912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:21.045921 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:38:21.046372 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:21.046373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:21.046384 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:21.046454 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:21.047078 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.047600 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.048353 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.048563 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.048643 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.051383 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:21.051385 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:21.051385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:21.051393 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:21.053204 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.053605 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.053606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.053606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.053665 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.053897 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.053898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.053899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.054043 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:21.054249 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:21.054287 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 06:38:21.054367 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 06:38:21.055107 17143 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:38:21.055199 17143 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:38:21.055445 17017 master_runner.cc:186] Error getting master registration for 127.29.142.190:39541: OK, instance_id { permanent_uuid: "813bbedb07e14129940c690e0553d223" instance_seqno: 1785998300664754 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:38:21.057758 17017 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:38:21.058744 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:38:21.069993 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.060836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54924 (local address 127.29.142.188:40241)
0806 06:38:21.060876 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:21.060878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:21.061098 (+ 220us) server_negotiation.cc:407] Connection header received
0806 06:38:21.061198 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:21.061199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:21.061210 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:21.061242 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:21.061988 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.062787 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.063963 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.064287 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.064323 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.066539 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:21.066541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:21.066541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:21.066550 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:21.068452 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.068988 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.068989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.068990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.069054 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.069308 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.069309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.069310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.069502 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:21.069570 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:21.069795 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 06:38:21.069890 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:38:21.069993 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.060425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37154 (local address 127.29.142.190:39541)
0806 06:38:21.060507 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:21.060510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:21.060805 (+ 295us) server_negotiation.cc:407] Connection header received
0806 06:38:21.060937 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:21.060939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:21.060953 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:21.060988 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:21.061976 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.062812 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.063963 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.064268 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.064318 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.066540 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:21.066541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:21.066541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:21.066550 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:21.068575 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.069013 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.069014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.069015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.069072 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.069303 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.069304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.069304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.069448 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:21.069529 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:21.069802 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 06:38:21.069889 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:38:21.070459 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.060665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57672 (local address 127.29.142.189:43715)
0806 06:38:21.060715 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:21.060718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:21.060949 (+ 231us) server_negotiation.cc:407] Connection header received
0806 06:38:21.061010 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:21.061011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:21.061022 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:21.061089 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:21.061964 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.062777 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.064027 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.064320 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.064366 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.066590 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:21.066592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:21.066592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:21.066600 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:21.069189 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.069599 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.069600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.069601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.069656 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.069873 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.069874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.069874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.070004 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:21.070071 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:21.070276 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 06:38:21.070359 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:38:21.072923 17008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } attempt: 3
Aug 06 06:38:21 dist-test-slave-3kk6 krb5kdc[16783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.189@KRBTEST.COM for kudu/127.29.142.188@KRBTEST.COM
I20260806 06:38:21.080698 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.073215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54932)
0806 06:38:21.073265 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:21.073276 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:21.073311 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:21.073464 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:21.073464 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:21.073633 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:21.073680 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:21.073681 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.074310 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.074311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:21.075158 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:21.075160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.075720 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.075721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:21.075786 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.076287 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:21.076292 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:21.077741 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:21.079445 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:21.079446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:21.079446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:21.079675 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:21.080055 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:21.080056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:21.080057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:21.080138 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:21.080436 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:21.080436 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:21.080478 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:38:21.080534 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:38:21.080624 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:38:21.080770 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.073300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54932 (local address 127.29.142.188:40241)
0806 06:38:21.073338 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:21.073340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:21.073348 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:38:21.073369 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:21.073369 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:21.073379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:21.073436 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:21.073750 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.074225 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.075340 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.075637 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.075704 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.077838 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:21.077840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:21.077840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:21.077847 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:21.079349 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.079771 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.079772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.079773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.079996 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.080214 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.080215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.080215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.080364 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:21.080439 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:21.080582 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 06:38:21.080670 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:38:21.083037 17008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6: Bootstrap starting.
I20260806 06:38:21.083897 17008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:21.084210 17008 log.cc:824] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:21.085253 17008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6: No bootstrap required, opened a new log
I20260806 06:38:21.087401 17008 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } }
I20260806 06:38:21.087565 17008 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:21.087605 17008 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97bf62a7f7c54edab719a941fc9b19b6, State: Initialized, Role: FOLLOWER
I20260806 06:38:21.087733 17008 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } }
I20260806 06:38:21.088064 17157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } } }
I20260806 06:38:21.088107 17008 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:38:21.088188 17157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:38:21.090167 17173 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:38:21.090276 17173 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:38:21.090440 16915 master_runner.cc:186] Error getting master registration for 127.29.142.190:39541: OK, instance_id { permanent_uuid: "813bbedb07e14129940c690e0553d223" instance_seqno: 1785998300664754 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:38:21.093474 16915 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:38:21.109743 16898 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } attempt: 3
Aug 06 06:38:21 dist-test-slave-3kk6 krb5kdc[16783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998300, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for kudu/127.29.142.188@KRBTEST.COM
I20260806 06:38:21.119451 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.110328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54934 (local address 127.29.142.188:40241)
0806 06:38:21.110450 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:21.110452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:21.110462 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:38:21.110493 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:21.110494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:21.110505 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:21.110554 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:21.111050 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.111839 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.112932 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.113277 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.113356 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.116164 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:21.116166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:21.116166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:21.116175 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:21.118020 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.118504 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.118505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.118506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.118576 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:21.118835 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:21.118836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:21.118837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:21.118996 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:21.119070 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:21.119259 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 06:38:21.119349 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 06:38:21.119453 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:21.110230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:40241 (local address 127.0.0.1:54934)
0806 06:38:21.110299 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:21.110314 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:21.110365 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:21.110609 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:21.110610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:21.110890 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:21.110951 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:21.110954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.111963 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.111965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:21.112814 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:21.112816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:21.113370 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:21.113372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:21.113468 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:21.114101 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:21.114108 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:21.116032 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:21.118135 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:21.118137 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:21.118137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:21.118406 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:21.118681 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:21.118682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:21.118683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:21.118759 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:21.119075 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:21.119076 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:21.119130 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 06:38:21.119199 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:38:21.119341 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:38:21.121630 16898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223: Bootstrap starting.
I20260806 06:38:21.122447 16898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:21.122843 16898 log.cc:824] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:21.123775 16898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223: No bootstrap required, opened a new log
I20260806 06:38:21.125888 16898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } }
I20260806 06:38:21.126057 16898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:21.126099 16898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 813bbedb07e14129940c690e0553d223, State: Initialized, Role: FOLLOWER
I20260806 06:38:21.126235 16898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } }
I20260806 06:38:21.126585 17178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } } }
I20260806 06:38:21.126643 16898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:38:21.128232 17178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:38:21.128980 17194 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:38:21.129057 17194 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:38:21.131560 16806 master_runner.cc:263] No leader master found from master 813bbedb07e14129940c690e0553d223
I20260806 06:38:21.131628 16806 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 06:38:21.162581 16835 tablet.cc:2369] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:38:21.200696 17157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:38:21.200857 17157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } }
I20260806 06:38:21.201140 17157 leader_election.cc:288] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 813bbedb07e14129940c690e0553d223 (127.29.142.190:39541), 9fab03e056174e3aa02e40e1a9c4846d (127.29.142.188:40241)
I20260806 06:38:21.201925 16863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97bf62a7f7c54edab719a941fc9b19b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "813bbedb07e14129940c690e0553d223" is_pre_election: true
I20260806 06:38:21.202092 16863 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97bf62a7f7c54edab719a941fc9b19b6 in term 0.
I20260806 06:38:21.202253 17075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97bf62a7f7c54edab719a941fc9b19b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fab03e056174e3aa02e40e1a9c4846d" is_pre_election: true
I20260806 06:38:21.202420 17075 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97bf62a7f7c54edab719a941fc9b19b6 in term 0.
I20260806 06:38:21.202955 16936 leader_election.cc:302] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 813bbedb07e14129940c690e0553d223, 97bf62a7f7c54edab719a941fc9b19b6; no voters:
I20260806 06:38:21.203238 17157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:38:21.203301 17157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:38:21.203325 17157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:21.204399 17157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } }
I20260806 06:38:21.204531 17157 leader_election.cc:288] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [CANDIDATE]: Term 1 election: Requested vote from peers 813bbedb07e14129940c690e0553d223 (127.29.142.190:39541), 9fab03e056174e3aa02e40e1a9c4846d (127.29.142.188:40241)
I20260806 06:38:21.204947 17075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97bf62a7f7c54edab719a941fc9b19b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fab03e056174e3aa02e40e1a9c4846d"
I20260806 06:38:21.205034 17075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:21.205862 17075 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97bf62a7f7c54edab719a941fc9b19b6 in term 1.
I20260806 06:38:21.206099 16863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97bf62a7f7c54edab719a941fc9b19b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "813bbedb07e14129940c690e0553d223"
I20260806 06:38:21.206192 16863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:21.207572 16863 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97bf62a7f7c54edab719a941fc9b19b6 in term 1.
I20260806 06:38:21.207818 16936 leader_election.cc:302] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 813bbedb07e14129940c690e0553d223, 97bf62a7f7c54edab719a941fc9b19b6; no voters:
I20260806 06:38:21.207969 17157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:38:21.208120 17157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [term 1 LEADER]: Becoming Leader. State: Replica: 97bf62a7f7c54edab719a941fc9b19b6, State: Running, Role: LEADER
I20260806 06:38:21.208207 17157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } }
I20260806 06:38:21.208597 17157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97bf62a7f7c54edab719a941fc9b19b6. Latest consensus state: current_term: 1 leader_uuid: "97bf62a7f7c54edab719a941fc9b19b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } } }
I20260806 06:38:21.208673 17157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:21.209815 17201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:38:21.210114 17201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:38:21.212446 17075 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [term 1 FOLLOWER]: Refusing update from remote peer 97bf62a7f7c54edab719a941fc9b19b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:38:21.212845 17157 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:38:21.214253 17128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97bf62a7f7c54edab719a941fc9b19b6. Latest consensus state: current_term: 1 leader_uuid: "97bf62a7f7c54edab719a941fc9b19b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } } }
I20260806 06:38:21.214347 17128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:38:21.214892 16863 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [term 1 FOLLOWER]: Refusing update from remote peer 97bf62a7f7c54edab719a941fc9b19b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:38:21.215226 17197 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:38:21.216233 17197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97bf62a7f7c54edab719a941fc9b19b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } } }
I20260806 06:38:21.216343 17197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:21.216611 17178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97bf62a7f7c54edab719a941fc9b19b6. Latest consensus state: current_term: 1 leader_uuid: "97bf62a7f7c54edab719a941fc9b19b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } } }
I20260806 06:38:21.216706 17178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:38:21.216748 17128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97bf62a7f7c54edab719a941fc9b19b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } } }
I20260806 06:38:21.216849 17128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:38:21.217068 17202 mvcc.cc:205] Tried to move back new op lower bound from 7315449041763708928 to 7315449041749352448. Current Snapshot: MvccSnapshot[applied={T|T < 7315449041763708928}]
I20260806 06:38:21.217662 17157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97bf62a7f7c54edab719a941fc9b19b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } } }
I20260806 06:38:21.217759 17157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:21.218003 17204 mvcc.cc:205] Tried to move back new op lower bound from 7315449041763708928 to 7315449041749352448. Current Snapshot: MvccSnapshot[applied={T|T < 7315449041763708928}]
I20260806 06:38:21.218113 17178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97bf62a7f7c54edab719a941fc9b19b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "813bbedb07e14129940c690e0553d223" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39541 } } peers { permanent_uuid: "97bf62a7f7c54edab719a941fc9b19b6" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 43715 } } peers { permanent_uuid: "9fab03e056174e3aa02e40e1a9c4846d" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 40241 } } }
I20260806 06:38:21.218196 17178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:38:21.218670 17201 catalog_manager.cc:1347] Generated new cluster ID: 013d17f927d04f96bdf20aecfe4c486c
I20260806 06:38:21.218721 17201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:38:21.219355 17208 mvcc.cc:205] Tried to move back new op lower bound from 7315449041763708928 to 7315449041749352448. Current Snapshot: MvccSnapshot[applied={T|T < 7315449041763708928}]
I20260806 06:38:21.235082 17201 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:38:21.235677 17201 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:38:21.247771 17201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97bf62a7f7c54edab719a941fc9b19b6: Generated new TSK 0
I20260806 06:38:21.248057 17201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:38:22.055711 17143 catalog_manager.cc:1259] Loaded cluster ID: 013d17f927d04f96bdf20aecfe4c486c
I20260806 06:38:22.055837 17143 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d: loading cluster ID for follower catalog manager: success
I20260806 06:38:22.056675 17143 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d: acquiring CA information for follower catalog manager: success
I20260806 06:38:22.057006 17143 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9fab03e056174e3aa02e40e1a9c4846d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:38:22.129585 17194 catalog_manager.cc:1259] Loaded cluster ID: 013d17f927d04f96bdf20aecfe4c486c
I20260806 06:38:22.129681 17194 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223: loading cluster ID for follower catalog manager: success
I20260806 06:38:22.130584 17194 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223: acquiring CA information for follower catalog manager: success
I20260806 06:38:22.130963 17194 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 813bbedb07e14129940c690e0553d223: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:38:22.311666 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 16806
I20260806 06:38:22.324999 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 16915
I20260806 06:38:22.335414 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 17017
2026-08-06T06:38:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3155 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:38:22 dist-test-slave-3kk6 krb5kdc[17222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:38:22 dist-test-slave-3kk6 krb5kdc[17222](info): set up 2 sockets
Aug 06 06:38:22 dist-test-slave-3kk6 krb5kdc[17222](info): commencing operation
krb5kdc: starting...
W20260806 06:38:23.556634 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:38:23 dist-test-slave-3kk6 krb5kdc[17222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:38:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:38:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:38:23.681815 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:38361
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:46663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:38361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:23.766620 17238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:23.766924 17238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:23.766968 17238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:23.768141 17238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:38:23.768224 17238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:23.768250 17238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:23.768282 17238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:38:23.768330 17238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:38:23.769903 17238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:46663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:38361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:38361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:23.770222 17238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:23.770480 17238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:23.775640 17246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:23.775619 17249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:23.775643 17245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:23.775892 17238 server_base.cc:1034] running on GCE node
I20260806 06:38:23.776268 17238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:23.776583 17238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:23.777760 17238 hybrid_clock.cc:648] HybridClock initialized: now 1785998303777752 us; error 33 us; skew 500 ppm
Aug 06 06:38:23 dist-test-slave-3kk6 krb5kdc[17222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998303, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:23.780411 17238 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:38:23.780884 17238 webserver.cc:460] Webserver started at http://127.29.142.190:43407/ using document root <none> and password file <none>
I20260806 06:38:23.781150 17238 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:23.781224 17238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:23.781383 17238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:38:23.782485 17238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "1a8a09eb5c29425db5edfda1cac88015"
format_stamp: "Formatted at 2026-08-06 06:38:23 on dist-test-slave-3kk6"
server_key: "3b962c7a182f780e5f3f0691403f5475"
server_key_iv: "0f6d00d0ec34f57836adfd2ffc4bc9b3"
server_key_version: "encryptionkey@0"
I20260806 06:38:23.782886 17238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "1a8a09eb5c29425db5edfda1cac88015"
format_stamp: "Formatted at 2026-08-06 06:38:23 on dist-test-slave-3kk6"
server_key: "3b962c7a182f780e5f3f0691403f5475"
server_key_iv: "0f6d00d0ec34f57836adfd2ffc4bc9b3"
server_key_version: "encryptionkey@0"
I20260806 06:38:23.784281 17238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:38:23.785019 17258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:23.785202 17238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:23.785323 17238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "1a8a09eb5c29425db5edfda1cac88015"
format_stamp: "Formatted at 2026-08-06 06:38:23 on dist-test-slave-3kk6"
server_key: "3b962c7a182f780e5f3f0691403f5475"
server_key_iv: "0f6d00d0ec34f57836adfd2ffc4bc9b3"
server_key_version: "encryptionkey@0"
I20260806 06:38:23.785403 17238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:23.818040 17238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:23.820770 17238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:23.820947 17238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:23.825387 17238 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:38361
I20260806 06:38:23.825428 17333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:38361 every 8 connection(s)
I20260806 06:38:23.825927 17238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:38:23.826943 17334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:23.827461 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 17238
I20260806 06:38:23.827582 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:38:23.827793 30266 external_mini_cluster.cc:1428] Setting key 11bc06503205522475152cbb6a157e5f
I20260806 06:38:23.830288 17334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015: Bootstrap starting.
I20260806 06:38:23.831054 17334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:23.831499 17334 log.cc:824] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:23.832293 17334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015: No bootstrap required, opened a new log
I20260806 06:38:23.834503 17334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8a09eb5c29425db5edfda1cac88015" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38361 } }
I20260806 06:38:23.834694 17334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:23.834751 17334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a8a09eb5c29425db5edfda1cac88015, State: Initialized, Role: FOLLOWER
I20260806 06:38:23.834905 17334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8a09eb5c29425db5edfda1cac88015" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38361 } }
I20260806 06:38:23.835011 17334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:38:23.835057 17334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:38:23.835111 17334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 06:38:23 dist-test-slave-3kk6 krb5kdc[17222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:23.835806 17334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8a09eb5c29425db5edfda1cac88015" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38361 } }
I20260806 06:38:23.835937 17334 leader_election.cc:302] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a8a09eb5c29425db5edfda1cac88015; no voters:
I20260806 06:38:23.836122 17334 leader_election.cc:288] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:38:23.836333 17341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:38:23.836416 17334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:38:23.836509 17341 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [term 1 LEADER]: Becoming Leader. State: Replica: 1a8a09eb5c29425db5edfda1cac88015, State: Running, Role: LEADER
I20260806 06:38:23.836647 17341 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8a09eb5c29425db5edfda1cac88015" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38361 } }
I20260806 06:38:23.837050 17341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a8a09eb5c29425db5edfda1cac88015. Latest consensus state: current_term: 1 leader_uuid: "1a8a09eb5c29425db5edfda1cac88015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8a09eb5c29425db5edfda1cac88015" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38361 } } }
I20260806 06:38:23.837306 17341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:23.837226 17340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a8a09eb5c29425db5edfda1cac88015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a8a09eb5c29425db5edfda1cac88015" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38361 } } }
I20260806 06:38:23.837597 17340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:23.837608 17355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:38:23.838260 17355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:38:23.839799 17337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:23.828413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58892 (local address 127.29.142.190:38361)
0806 06:38:23.828614 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:23.828625 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:23.828661 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:38:23.829221 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:23.829230 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:23.829290 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:23.829447 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:23.830069 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:23.831534 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:23.832534 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:23.833146 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:23.833274 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:23.835775 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:23.835779 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:23.835780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:23.835792 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:23.837964 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:23.838478 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:23.838481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:23.838481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:23.838600 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:23.838883 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:23.838884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:23.838884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:23.839189 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:23.839340 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:23.839484 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 06:38:23.839630 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260806 06:38:23.840242 17355 catalog_manager.cc:1347] Generated new cluster ID: a547716f19aa4ab1a873157cf90d93c7
I20260806 06:38:23.840294 17355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:38:23.840730 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:38:23.848237 17355 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:38:23.848995 17355 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:38:23.861153 17355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a8a09eb5c29425db5edfda1cac88015: Generated new TSK 0
I20260806 06:38:23.861390 17355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:38:23.894883 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 17238
2026-08-06T06:38:23Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1558 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:38:23 dist-test-slave-3kk6 krb5kdc[17370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:38:23 dist-test-slave-3kk6 krb5kdc[17370](info): set up 2 sockets
Aug 06 06:38:23 dist-test-slave-3kk6 krb5kdc[17370](info): commencing operation
krb5kdc: starting...
W20260806 06:38:25.109889 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:38:25 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:38:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:38:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:38:25.237195 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:33889
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:33889
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:25.321667 17384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:25.321940 17384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:25.321977 17384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:25.323159 17384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:38:25.323235 17384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:25.323259 17384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:25.323280 17384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:38:25.323310 17384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:38:25.324904 17384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:33889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:33889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:25.325196 17384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:25.325441 17384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:25.330632 17390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:25.330624 17393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:25.330864 17384 server_base.cc:1034] running on GCE node
W20260806 06:38:25.330605 17389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:25.331156 17384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:25.331511 17384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:25.332674 17384 hybrid_clock.cc:648] HybridClock initialized: now 1785998305332658 us; error 30 us; skew 500 ppm
Aug 06 06:38:25 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998305, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:25.335404 17384 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:38:25.335898 17384 webserver.cc:460] Webserver started at http://127.29.142.190:38995/ using document root <none> and password file <none>
I20260806 06:38:25.336154 17384 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:25.336222 17384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:25.336372 17384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:38:25.337466 17384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "769bebc24e8149b3a2593701f1feb123"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "81d0027dcf946a8b3e920c8ce060e1dd"
server_key_iv: "d65ec870dc18e158928c1dde37368a66"
server_key_version: "encryptionkey@0"
I20260806 06:38:25.337838 17384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "769bebc24e8149b3a2593701f1feb123"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "81d0027dcf946a8b3e920c8ce060e1dd"
server_key_iv: "d65ec870dc18e158928c1dde37368a66"
server_key_version: "encryptionkey@0"
I20260806 06:38:25.339303 17384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:38:25.340111 17403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:25.340301 17384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:25.340407 17384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "769bebc24e8149b3a2593701f1feb123"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "81d0027dcf946a8b3e920c8ce060e1dd"
server_key_iv: "d65ec870dc18e158928c1dde37368a66"
server_key_version: "encryptionkey@0"
I20260806 06:38:25.340483 17384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:25.353379 17384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:25.355872 17384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:25.356051 17384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:25.360250 17384 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:33889
I20260806 06:38:25.360327 17479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:33889 every 8 connection(s)
I20260806 06:38:25.360699 17384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:38:25.361799 17480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:25.362890 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 17384
I20260806 06:38:25.363009 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:38:25.363204 30266 external_mini_cluster.cc:1428] Setting key abfa2857e5be40a114b826a6ca4acbf7
I20260806 06:38:25.364771 17480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Bootstrap starting.
I20260806 06:38:25.365548 17480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:25.365859 17480 log.cc:824] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:25.366629 17480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: No bootstrap required, opened a new log
I20260806 06:38:25.369050 17480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } }
I20260806 06:38:25.369266 17480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:25.369309 17480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 769bebc24e8149b3a2593701f1feb123, State: Initialized, Role: FOLLOWER
I20260806 06:38:25.369462 17480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } }
I20260806 06:38:25.369562 17480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:38:25.369601 17480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:38:25.369660 17480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 06:38:25 dist-test-slave-3kk6 krb5kdc[17370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:25.370253 17480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } }
I20260806 06:38:25.370366 17480 leader_election.cc:302] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 769bebc24e8149b3a2593701f1feb123; no voters:
I20260806 06:38:25.370581 17480 leader_election.cc:288] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:38:25.370602 17488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:38:25.370746 17488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 1 LEADER]: Becoming Leader. State: Replica: 769bebc24e8149b3a2593701f1feb123, State: Running, Role: LEADER
I20260806 06:38:25.370896 17480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:38:25.370904 17488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } }
I20260806 06:38:25.371309 17488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 769bebc24e8149b3a2593701f1feb123. Latest consensus state: current_term: 1 leader_uuid: "769bebc24e8149b3a2593701f1feb123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } } }
I20260806 06:38:25.371429 17488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:25.371573 17491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "769bebc24e8149b3a2593701f1feb123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } } }
I20260806 06:38:25.371690 17499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:38:25.371739 17491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:25.372256 17499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:38:25.374002 17499 catalog_manager.cc:1347] Generated new cluster ID: 8f935dede7e44853933a08c9aae6d376
I20260806 06:38:25.374055 17499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:38:25.374804 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.363782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50842 (local address 127.29.142.190:33889)
0806 06:38:25.363967 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:25.363976 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:25.364011 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:38:25.364450 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:25.364457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:25.364501 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:25.364617 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:25.365475 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.366478 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.367549 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.367965 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.368055 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.370892 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:25.370895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:25.370896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:25.370909 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:25.372985 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:25.373432 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:25.373436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:25.373436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:25.373545 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:25.373809 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:25.373811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:25.373811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:25.374135 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:25.374300 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:25.374424 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 06:38:25.374571 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260806 06:38:25.399266 17499 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:38:25.400061 17499 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:38:25.406261 17499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Generated new TSK 0
I20260806 06:38:25.406504 17499 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:38:25.441496 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:33889
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:25.527359 17519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:25.527567 17519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:25.527591 17519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:25.528815 17519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:25.528861 17519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:25.528889 17519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:38:25.530308 17519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:25.530514 17519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:25.530735 17519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:25.535895 17527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:25.536006 17519 server_base.cc:1034] running on GCE node
W20260806 06:38:25.536180 17529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:25.536175 17526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:25.536480 17519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:25.536752 17519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:25.537964 17519 hybrid_clock.cc:648] HybridClock initialized: now 1785998305537941 us; error 42 us; skew 500 ppm
Aug 06 06:38:25 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998305, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:25.540784 17519 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:38:25.541260 17519 webserver.cc:460] Webserver started at http://127.29.142.129:40315/ using document root <none> and password file <none>
I20260806 06:38:25.541524 17519 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:25.541597 17519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:25.541756 17519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:38:25.543040 17519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "bc4a424401414d848570034b484651a3"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "e8f9295c2088cec45d0b24c4990be8d4"
server_key_iv: "ca12254674a4f16a3f1404a71d22eace"
server_key_version: "encryptionkey@0"
I20260806 06:38:25.543470 17519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "bc4a424401414d848570034b484651a3"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "e8f9295c2088cec45d0b24c4990be8d4"
server_key_iv: "ca12254674a4f16a3f1404a71d22eace"
server_key_version: "encryptionkey@0"
I20260806 06:38:25.544996 17519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:38:25.545781 17539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:25.545979 17519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:25.546090 17519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "bc4a424401414d848570034b484651a3"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "e8f9295c2088cec45d0b24c4990be8d4"
server_key_iv: "ca12254674a4f16a3f1404a71d22eace"
server_key_version: "encryptionkey@0"
I20260806 06:38:25.546171 17519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:25.558940 17519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:25.561316 17519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:25.561523 17519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:25.561797 17519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:25.562153 17519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:25.562213 17519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:25.562249 17519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:25.562279 17519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:25.568518 17519 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:42053
I20260806 06:38:25.568625 17688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:42053 every 8 connection(s)
I20260806 06:38:25.568943 17519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:38:25 dist-test-slave-3kk6 krb5kdc[17370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998305, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:25.577661 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 17519
I20260806 06:38:25.577822 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:38:25.578085 30266 external_mini_cluster.cc:1428] Setting key c2d303760aa2e4ee77210eeeb321c2fe
I20260806 06:38:25.579146 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.569698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:42257 (local address 127.29.142.190:33889)
0806 06:38:25.569769 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:25.569772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:25.570039 (+ 267us) server_negotiation.cc:407] Connection header received
0806 06:38:25.570547 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:25.570548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:25.570559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:25.570616 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:25.571676 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.572370 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.573369 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.573577 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.573637 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.576011 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:25.576012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:25.576012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:25.576020 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:25.577351 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:25.577927 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:25.577929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:25.577929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:25.577996 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:25.578330 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:25.578331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:25.578332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:25.578474 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:25.578530 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:25.578993 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 06:38:25.579081 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:38:25.579308 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.569669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:42257)
0806 06:38:25.569877 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:25.569935 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:25.570451 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:25.570731 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:25.570735 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:25.571364 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:25.571557 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:25.571562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.572454 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.572455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:25.573278 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:25.573280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.573656 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.573656 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:25.573795 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.574393 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:25.574402 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:25.575905 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:25.577484 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:25.577493 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:25.577495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:25.577828 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:25.578073 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:25.578074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:25.578074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:25.578237 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:25.578526 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:25.578534 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:25.578727 (+ 193us) client_negotiation.cc:769] Sending connection context
0806 06:38:25.578855 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 06:38:25.579030 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":118,"threads_started":1}
I20260806 06:38:25.579818 17689 heartbeater.cc:347] Connected to a master server at 127.29.142.190:33889
I20260806 06:38:25.579932 17689 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:25.580128 17689 heartbeater.cc:511] Master 127.29.142.190:33889 requested a full tablet report, sending...
I20260806 06:38:25.580736 17431 ts_manager.cc:194] Registered new tserver with Master: bc4a424401414d848570034b484651a3 (127.29.142.129:42053)
I20260806 06:38:25.581355 17431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:42257
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:38:25.606555 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:33889
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:25.692780 17696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:25.693055 17696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:25.693099 17696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:25.694289 17696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:25.694362 17696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:25.694408 17696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:38:25.695956 17696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:25.696257 17696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:25.696506 17696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:38:25.701777 17696 server_base.cc:1034] running on GCE node
W20260806 06:38:25.701898 17705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:25.701952 17701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:25.702011 17702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:25.702236 17696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:25.702447 17696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:25.703617 17696 hybrid_clock.cc:648] HybridClock initialized: now 1785998305703585 us; error 47 us; skew 500 ppm
Aug 06 06:38:25 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998305, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:25.712730 17696 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:38:25.713290 17696 webserver.cc:460] Webserver started at http://127.29.142.130:39301/ using document root <none> and password file <none>
I20260806 06:38:25.713544 17696 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:25.713617 17696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:25.713783 17696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:38:25.715021 17696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "f503992182b24a079f7162ffdd66308d"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "b2adcac1dccbe64ee363f98894989992"
server_key_iv: "7e852272d0d1bc53733b5edeaeac1c4e"
server_key_version: "encryptionkey@0"
I20260806 06:38:25.715420 17696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "f503992182b24a079f7162ffdd66308d"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "b2adcac1dccbe64ee363f98894989992"
server_key_iv: "7e852272d0d1bc53733b5edeaeac1c4e"
server_key_version: "encryptionkey@0"
I20260806 06:38:25.716920 17696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:38:25.717868 17717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:25.718099 17696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:25.718225 17696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "f503992182b24a079f7162ffdd66308d"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "b2adcac1dccbe64ee363f98894989992"
server_key_iv: "7e852272d0d1bc53733b5edeaeac1c4e"
server_key_version: "encryptionkey@0"
I20260806 06:38:25.718312 17696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:25.738090 17696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:25.740762 17696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:25.740952 17696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:25.741241 17696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:25.741614 17696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:25.741670 17696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:25.741707 17696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:25.741739 17696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:25.748411 17696 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:35759
I20260806 06:38:25.748463 17872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:35759 every 8 connection(s)
I20260806 06:38:25.748857 17696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:38:25.752573 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 17696
I20260806 06:38:25.752733 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:38:25.752972 30266 external_mini_cluster.cc:1428] Setting key 9887e0ebf6e1cc64c949d3a2beb2b3b8
Aug 06 06:38:25 dist-test-slave-3kk6 krb5kdc[17370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998305, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:25.760808 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.749805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:36359 (local address 127.29.142.190:33889)
0806 06:38:25.749872 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:25.749875 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:25.750082 (+ 207us) server_negotiation.cc:407] Connection header received
0806 06:38:25.750811 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:25.750812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:25.750825 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:25.750886 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:25.751940 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.752460 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.754127 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.754335 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.754395 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.757440 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:25.757442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:25.757442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:25.757456 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:25.759075 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:25.759686 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:25.759687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:25.759687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:25.759744 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:25.760083 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:25.760084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:25.760084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:25.760194 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:25.760255 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:25.760663 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 06:38:25.760736 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:38:25.761176 17877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.749777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:36359)
0806 06:38:25.749956 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:25.749997 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:25.750702 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:25.750948 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:25.750951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:25.751597 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:25.751833 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:25.751839 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.752592 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.752594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:25.754024 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:25.754027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.754436 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.754437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:25.754620 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.755331 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:25.755344 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:25.757325 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:25.759171 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:25.759188 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:25.759190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:25.759593 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:25.759822 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:25.759823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:25.759824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:25.760008 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:25.760278 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:25.760286 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:25.760503 (+ 217us) client_negotiation.cc:769] Sending connection context
0806 06:38:25.760652 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 06:38:25.760880 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260806 06:38:25.761860 17873 heartbeater.cc:347] Connected to a master server at 127.29.142.190:33889
I20260806 06:38:25.761993 17873 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:25.762250 17873 heartbeater.cc:511] Master 127.29.142.190:33889 requested a full tablet report, sending...
I20260806 06:38:25.762763 17431 ts_manager.cc:194] Registered new tserver with Master: f503992182b24a079f7162ffdd66308d (127.29.142.130:35759)
I20260806 06:38:25.763154 17431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:36359
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:38:25.783594 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:33889
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:25.870288 17880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:25.870517 17880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:25.870539 17880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:25.871891 17880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:25.871937 17880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:25.871964 17880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:38:25.873479 17880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:25.873672 17880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:25.873865 17880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:25.879098 17896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:25.879099 17894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:25.879101 17892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:25.879323 17880 server_base.cc:1034] running on GCE node
I20260806 06:38:25.879659 17880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:25.879925 17880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:25.881114 17880 hybrid_clock.cc:648] HybridClock initialized: now 1785998305881103 us; error 32 us; skew 500 ppm
Aug 06 06:38:25 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998305, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:25.883852 17880 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:38:25.884338 17880 webserver.cc:460] Webserver started at http://127.29.142.131:34663/ using document root <none> and password file <none>
I20260806 06:38:25.884557 17880 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:25.884600 17880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:25.884694 17880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:38:25.885814 17880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "4d98aeb86d97444bbeb1ef704adb3816"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "a2a6f682de4d3ccb0981c9c4659451e6"
server_key_iv: "c9c01f73fee855ae1ac6c197b9bc71f7"
server_key_version: "encryptionkey@0"
I20260806 06:38:25.886166 17880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "4d98aeb86d97444bbeb1ef704adb3816"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "a2a6f682de4d3ccb0981c9c4659451e6"
server_key_iv: "c9c01f73fee855ae1ac6c197b9bc71f7"
server_key_version: "encryptionkey@0"
I20260806 06:38:25.887550 17880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:38:25.888288 17908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:25.888495 17880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:38:25.888593 17880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "4d98aeb86d97444bbeb1ef704adb3816"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "a2a6f682de4d3ccb0981c9c4659451e6"
server_key_iv: "c9c01f73fee855ae1ac6c197b9bc71f7"
server_key_version: "encryptionkey@0"
I20260806 06:38:25.888653 17880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:25.897332 17880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:25.899761 17880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:25.899885 17880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:25.900096 17880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:25.900365 17880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:25.900399 17880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:25.900421 17880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:25.900434 17880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:25.906415 17880 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:34803
I20260806 06:38:25.906498 18066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:34803 every 8 connection(s)
I20260806 06:38:25.906890 17880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:38:25.909654 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 17880
I20260806 06:38:25.909828 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:38:25.910132 30266 external_mini_cluster.cc:1428] Setting key 888cdca8f46716e123abe3ee4fbe7bcc
Aug 06 06:38:25 dist-test-slave-3kk6 krb5kdc[17370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998305, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:25.918632 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.907864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:54327 (local address 127.29.142.190:33889)
0806 06:38:25.907924 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:25.907927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:25.908051 (+ 124us) server_negotiation.cc:407] Connection header received
0806 06:38:25.909036 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:25.909038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:25.909049 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:25.909098 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:25.910720 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.911512 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.912744 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.912941 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.913009 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.915603 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:25.915604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:25.915604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:25.915612 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:25.917193 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:25.917615 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:25.917616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:25.917616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:25.917709 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:25.917947 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:25.917948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:25.917948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:25.918076 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:25.918130 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:25.918423 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 06:38:25.918500 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:38:25.918731 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.907695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:54327)
0806 06:38:25.907888 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:25.907941 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:25.908900 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:25.909245 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:25.909273 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:25.910155 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:25.910505 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:25.910514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.911798 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.911799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:25.912632 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:25.912634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.913026 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.913027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:25.913134 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.913856 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:25.913865 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:25.915434 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:25.917285 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:25.917292 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:25.917294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:25.917523 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:25.917779 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:25.917780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:25.917780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:25.917885 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:25.918132 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:25.918137 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:25.918268 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 06:38:25.918355 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:38:25.918486 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
I20260806 06:38:25.919260 18067 heartbeater.cc:347] Connected to a master server at 127.29.142.190:33889
I20260806 06:38:25.919380 18067 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:25.919605 18067 heartbeater.cc:511] Master 127.29.142.190:33889 requested a full tablet report, sending...
I20260806 06:38:25.920038 17431 ts_manager.cc:194] Registered new tserver with Master: 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803)
I20260806 06:38:25.920383 17431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:54327
I20260806 06:38:25.922302 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:38:25.929867 17485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.923683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50850 (local address 127.29.142.190:33889)
0806 06:38:25.923777 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:25.923779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:25.923848 (+ 69us) server_negotiation.cc:407] Connection header received
0806 06:38:25.923898 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:25.923898 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:25.923907 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:25.923947 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:25.924507 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.925025 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.925709 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.925897 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.925953 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.927219 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:25.927220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:25.927220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:25.927226 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:25.928796 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:25.929109 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:25.929109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:25.929110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:25.929241 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:25.929413 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:25.929414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:25.929414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:25.929557 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:25.929634 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:25.929723 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 06:38:25.929780 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 06:38:25.931660 17431 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:38:25.933281 17431 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:38:25.940155 18082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.935690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:35759 (local address 127.0.0.1:40712)
0806 06:38:25.936011 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:25.936056 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:25.936126 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:25.936650 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:25.936668 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:25.936670 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:25.936718 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:25.936721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.937657 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.937658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:25.938910 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:25.938912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.939901 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.939903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:25.939972 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.939978 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:38:25.940009 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 06:38:25.940081 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":25,"thread_start_us":114,"threads_started":1}
I20260806 06:38:25.940166 18085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.935811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.29.142.129:42053)
0806 06:38:25.936463 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:25.936468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:25.936477 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:38:25.936517 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:25.936519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:25.936537 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:25.936619 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:25.936811 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.937892 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.939182 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.939690 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.939797 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.939868 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:25.940022 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 06:38:25.940069 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":109,"threads_started":1}
I20260806 06:38:25.940299 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.936206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40712 (local address 127.29.142.130:35759)
0806 06:38:25.936457 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:25.936462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:25.936474 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:38:25.936518 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:25.936519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:25.936538 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:25.936825 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:25.936901 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.937554 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.939001 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.939785 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.940060 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.940119 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:25.940145 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:38:25.940199 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":114,"threads_started":1}
I20260806 06:38:25.940497 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.935974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57084 (local address 127.29.142.131:34803)
0806 06:38:25.936185 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:25.936191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:25.936305 (+ 114us) server_negotiation.cc:407] Connection header received
0806 06:38:25.936386 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:25.936389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:25.936407 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:25.936483 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:25.936730 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.937908 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.939222 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.939933 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.939996 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.940052 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:25.940379 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 06:38:25.940426 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":108,"threads_started":1}
I20260806 06:38:25.940300 18084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.935692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:34803 (local address 127.0.0.1:57084)
0806 06:38:25.936233 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:25.936242 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:25.936311 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:25.936589 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:25.936592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:25.936595 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:25.936665 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:25.936667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.938026 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.938027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:25.939123 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:25.939126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.940087 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.940088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:25.940144 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.940148 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:38:25.940196 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:38:25.940232 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":19,"thread_start_us":117,"threads_started":1}
I20260806 06:38:25.940816 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:25.935688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42053 (local address 127.0.0.1:55188)
0806 06:38:25.936010 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:25.936054 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:25.936138 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:25.936702 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:25.936703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:25.936705 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:25.936742 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:25.936743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.938012 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.938013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:25.939102 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:25.939104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:25.939797 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:25.939798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:25.939886 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:25.939899 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:38:25.940002 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 06:38:25.940721 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":22,"thread_start_us":50,"threads_started":1}
I20260806 06:38:25.941385 17788 tablet_service.cc:1467] Processing CreateTablet for tablet 97ce1991bdfd468fa07d6f4a9202caec (DEFAULT_TABLE table=test-table [id=35b711488789481184da73cfbccc727a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:38:25.941656 17598 tablet_service.cc:1467] Processing CreateTablet for tablet 97ce1991bdfd468fa07d6f4a9202caec (DEFAULT_TABLE table=test-table [id=35b711488789481184da73cfbccc727a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:38:25.941656 17973 tablet_service.cc:1467] Processing CreateTablet for tablet 97ce1991bdfd468fa07d6f4a9202caec (DEFAULT_TABLE table=test-table [id=35b711488789481184da73cfbccc727a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:38:25.941795 17788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ce1991bdfd468fa07d6f4a9202caec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:25.942003 17973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ce1991bdfd468fa07d6f4a9202caec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:25.942003 17598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ce1991bdfd468fa07d6f4a9202caec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:25.945035 18088 tablet_bootstrap.cc:492] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Bootstrap starting.
I20260806 06:38:25.945446 18090 tablet_bootstrap.cc:492] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816: Bootstrap starting.
I20260806 06:38:25.945801 18088 tablet_bootstrap.cc:654] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:25.946072 18090 tablet_bootstrap.cc:654] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:25.946370 18088 log.cc:824] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:25.946588 18090 log.cc:824] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:25.946815 18089 tablet_bootstrap.cc:492] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Bootstrap starting.
I20260806 06:38:25.947395 18090 tablet_bootstrap.cc:492] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816: No bootstrap required, opened a new log
I20260806 06:38:25.947453 18090 ts_tablet_manager.cc:1397] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:38:25.947468 18089 tablet_bootstrap.cc:654] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:25.948212 18088 tablet_bootstrap.cc:492] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: No bootstrap required, opened a new log
I20260806 06:38:25.948318 18088 ts_tablet_manager.cc:1397] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:38:25.949141 18090 raft_consensus.cc:348] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:25.949286 18090 raft_consensus.cc:374] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:25.949309 18090 raft_consensus.cc:729] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d98aeb86d97444bbeb1ef704adb3816, State: Initialized, Role: FOLLOWER
I20260806 06:38:25.949438 18090 consensus_queue.cc:260] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:25.949769 18090 ts_tablet_manager.cc:1428] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 06:38:25.949828 18067 heartbeater.cc:503] Master 127.29.142.190:33889 was elected leader, sending a full tablet report...
I20260806 06:38:25.950224 18088 raft_consensus.cc:348] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:25.950380 18088 raft_consensus.cc:374] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:25.950404 18088 raft_consensus.cc:729] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f503992182b24a079f7162ffdd66308d, State: Initialized, Role: FOLLOWER
I20260806 06:38:25.950522 18088 consensus_queue.cc:260] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:25.950937 18089 log.cc:824] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:25.951634 17873 heartbeater.cc:503] Master 127.29.142.190:33889 was elected leader, sending a full tablet report...
I20260806 06:38:25.951784 18089 tablet_bootstrap.cc:492] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: No bootstrap required, opened a new log
I20260806 06:38:25.951880 18089 ts_tablet_manager.cc:1397] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 06:38:25.951618 18088 ts_tablet_manager.cc:1428] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:38:25.953498 18089 raft_consensus.cc:348] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:25.953648 18089 raft_consensus.cc:374] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:25.953697 18089 raft_consensus.cc:729] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc4a424401414d848570034b484651a3, State: Initialized, Role: FOLLOWER
I20260806 06:38:25.953819 18089 consensus_queue.cc:260] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:25.954092 18089 ts_tablet_manager.cc:1428] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:38:25.954147 17689 heartbeater.cc:503] Master 127.29.142.190:33889 was elected leader, sending a full tablet report...
W20260806 06:38:25.999559 17875 tablet.cc:2369] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:38:26.069825 17691 tablet.cc:2369] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:38:26.157521 18068 tablet.cc:2369] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:38:26.243829 18103 raft_consensus.cc:482] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:38:26.244048 18103 raft_consensus.cc:504] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:26.244482 18103 leader_election.cc:288] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f503992182b24a079f7162ffdd66308d (127.29.142.130:35759), 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803)
I20260806 06:38:26.248096 18105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.244718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:35759 (local address 127.29.142.129:36339)
0806 06:38:26.244885 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:26.244898 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:26.244976 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:26.245205 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:26.245206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:26.245209 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:26.245302 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.245304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.246123 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.246124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.247251 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.247254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.247879 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.247880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.247932 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.247935 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:38:26.247979 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:38:26.248010 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":77,"threads_started":1}
I20260806 06:38:26.248106 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.244799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:36339 (local address 127.29.142.130:35759)
0806 06:38:26.244866 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:26.244868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:26.244959 (+ 91us) server_negotiation.cc:407] Connection header received
0806 06:38:26.245079 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:26.245081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:26.245093 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:26.245166 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:26.245374 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.246038 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.247343 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.247792 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.247875 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.247885 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:26.248001 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 06:38:26.248036 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:38:26.248560 18106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.244760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:34803 (local address 127.29.142.129:54253)
0806 06:38:26.245410 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:26.245419 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:26.245467 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:26.245628 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:26.245629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:26.245630 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:26.245693 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.245695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.246526 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.246527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.247538 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.247540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.248348 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.248348 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.248411 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.248415 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:38:26.248462 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:38:26.248499 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":65,"threads_started":1}
I20260806 06:38:26.248503 17816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce1991bdfd468fa07d6f4a9202caec" candidate_uuid: "bc4a424401414d848570034b484651a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f503992182b24a079f7162ffdd66308d" is_pre_election: true
I20260806 06:38:26.248593 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.244853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:54253 (local address 127.29.142.131:34803)
0806 06:38:26.244900 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:26.244902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:26.245466 (+ 564us) server_negotiation.cc:407] Connection header received
0806 06:38:26.245536 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:26.245538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:26.245551 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:26.245609 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:26.245784 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.246433 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.247623 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.248235 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.248311 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.248319 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:26.248485 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 06:38:26.248531 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:38:26.248685 17816 raft_consensus.cc:2454] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc4a424401414d848570034b484651a3 in term 0.
I20260806 06:38:26.248981 17541 leader_election.cc:302] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc4a424401414d848570034b484651a3, f503992182b24a079f7162ffdd66308d; no voters:
I20260806 06:38:26.249002 18001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce1991bdfd468fa07d6f4a9202caec" candidate_uuid: "bc4a424401414d848570034b484651a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d98aeb86d97444bbeb1ef704adb3816" is_pre_election: true
I20260806 06:38:26.249145 18001 raft_consensus.cc:2454] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc4a424401414d848570034b484651a3 in term 0.
I20260806 06:38:26.249127 18103 raft_consensus.cc:2781] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:38:26.249208 18103 raft_consensus.cc:482] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:38:26.249238 18103 raft_consensus.cc:3037] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:26.250208 18103 raft_consensus.cc:504] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:26.250363 18103 leader_election.cc:288] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 1 election: Requested vote from peers f503992182b24a079f7162ffdd66308d (127.29.142.130:35759), 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803)
I20260806 06:38:26.250509 17816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce1991bdfd468fa07d6f4a9202caec" candidate_uuid: "bc4a424401414d848570034b484651a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f503992182b24a079f7162ffdd66308d"
I20260806 06:38:26.250562 18001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ce1991bdfd468fa07d6f4a9202caec" candidate_uuid: "bc4a424401414d848570034b484651a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d98aeb86d97444bbeb1ef704adb3816"
I20260806 06:38:26.250595 17816 raft_consensus.cc:3037] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:26.250674 18001 raft_consensus.cc:3037] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:26.251372 17816 raft_consensus.cc:2454] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc4a424401414d848570034b484651a3 in term 1.
I20260806 06:38:26.251477 18001 raft_consensus.cc:2454] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc4a424401414d848570034b484651a3 in term 1.
I20260806 06:38:26.251552 17541 leader_election.cc:302] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc4a424401414d848570034b484651a3, f503992182b24a079f7162ffdd66308d; no voters:
I20260806 06:38:26.251663 18103 raft_consensus.cc:2781] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:38:26.251874 18103 raft_consensus.cc:686] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 1 LEADER]: Becoming Leader. State: Replica: bc4a424401414d848570034b484651a3, State: Running, Role: LEADER
I20260806 06:38:26.251987 18103 consensus_queue.cc:237] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:26.252903 17431 catalog_manager.cc:5495] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc4a424401414d848570034b484651a3 (127.29.142.129). New cstate: current_term: 1 leader_uuid: "bc4a424401414d848570034b484651a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } health_report { overall_health: UNKNOWN } } }
I20260806 06:38:26.298303 18085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.295821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55200 (local address 127.29.142.129:42053)
0806 06:38:26.295890 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:26.295892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:26.295900 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:38:26.295949 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:26.295950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:26.295972 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:26.296059 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:38:26.296216 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.296702 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.297490 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.297699 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.297780 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.297942 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:38:26.298055 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:38:26.298158 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:26.298179 (+ 21us) server_negotiation.cc:299] Negotiation successful
0806 06:38:26.298201 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:38:26.300457 18001 raft_consensus.cc:1261] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816 [term 1 FOLLOWER]: Refusing update from remote peer bc4a424401414d848570034b484651a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:38:26.300457 17816 raft_consensus.cc:1261] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 1 FOLLOWER]: Refusing update from remote peer bc4a424401414d848570034b484651a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:38:26.300891 18103 consensus_queue.cc:1035] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:38:26.301064 18107 consensus_queue.cc:1035] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:38:26.303501 18110 mvcc.cc:205] Tried to move back new op lower bound from 7315449062603661312 to 7315449062408806400. Current Snapshot: MvccSnapshot[applied={T|T < 7315449062603661312}]
I20260806 06:38:26.303787 18111 mvcc.cc:205] Tried to move back new op lower bound from 7315449062603661312 to 7315449062408806400. Current Snapshot: MvccSnapshot[applied={T|T < 7315449062603661312}]
I20260806 06:38:26.303915 18112 mvcc.cc:205] Tried to move back new op lower bound from 7315449062603661312 to 7315449062408806400. Current Snapshot: MvccSnapshot[applied={T|T < 7315449062603661312}]
I20260806 06:38:26.306368 17428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:38:26.306480 17428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:38:26.307353 17428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Sending DeleteTablet for 3 replicas of tablet 97ce1991bdfd468fa07d6f4a9202caec
I20260806 06:38:26.307696 17788 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ce1991bdfd468fa07d6f4a9202caec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:26 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:40712
I20260806 06:38:26.307744 17598 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ce1991bdfd468fa07d6f4a9202caec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:26 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:55188
I20260806 06:38:26.307881 17973 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ce1991bdfd468fa07d6f4a9202caec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:26 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:57084
I20260806 06:38:26.308100 18127 tablet_replica.cc:330] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: stopping tablet replica
I20260806 06:38:26.308125 18128 tablet_replica.cc:330] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816: stopping tablet replica
I20260806 06:38:26.308141 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 17519
I20260806 06:38:26.311767 18128 raft_consensus.cc:2229] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:38:26.311951 18128 raft_consensus.cc:2258] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:38:26.314185 18128 ts_tablet_manager.cc:1905] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:38:26.316777 18128 ts_tablet_manager.cc:1918] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:38:26.316941 18128 log.cc:1196] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/97ce1991bdfd468fa07d6f4a9202caec
I20260806 06:38:26.317361 18128 ts_tablet_manager.cc:1939] T 97ce1991bdfd468fa07d6f4a9202caec P 4d98aeb86d97444bbeb1ef704adb3816: Deleting consensus metadata
I20260806 06:38:26.318853 17404 catalog_manager.cc:4841] TS 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803): tablet 97ce1991bdfd468fa07d6f4a9202caec (table test-table [id=35b711488789481184da73cfbccc727a]) successfully deleted
I20260806 06:38:26.322832 18126 tablet_replica.cc:330] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: stopping tablet replica
I20260806 06:38:26.323020 18126 raft_consensus.cc:2229] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:38:26.323174 18126 raft_consensus.cc:2258] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 06:38:26.323727 17407 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:42053 (error 108)
I20260806 06:38:26.323932 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 17696
I20260806 06:38:26.325965 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.324232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42053 (local address 127.0.0.1:55206)
0806 06:38:26.325883 (+ 1651us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:26.325927 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:42053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1627}
W20260806 06:38:26.326124 17407 catalog_manager.cc:4584] TS bc4a424401414d848570034b484651a3 (127.29.142.129:42053): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97ce1991bdfd468fa07d6f4a9202caec: Network error: Client connection negotiation failed: client connection to 127.29.142.129:42053: connect: Connection refused (error 111)
I20260806 06:38:26.336369 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 17880
I20260806 06:38:26.336417 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.336257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:35759 (local address 127.0.0.1:40718)
0806 06:38:26.336328 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:26.336368 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.130:35759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 06:38:26.352761 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 17384
I20260806 06:38:26.364377 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:33889
--webserver_interface=127.29.142.190
--webserver_port=38995
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:33889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:26.450274 18131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:26.450510 18131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:26.450534 18131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:26.451774 18131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:38:26.451826 18131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:26.451841 18131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:26.451854 18131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:38:26.451908 18131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:38:26.453410 18131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:33889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:33889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:26.453703 18131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:26.453951 18131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:26.463922 18145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:26.463964 18141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:26.463944 18131 server_base.cc:1034] running on GCE node
W20260806 06:38:26.463991 18142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:26.464397 18131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:26.464823 18131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:26.466018 18131 hybrid_clock.cc:648] HybridClock initialized: now 1785998306466013 us; error 40 us; skew 500 ppm
Aug 06 06:38:26 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:26.468792 18131 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:38:26.469224 18131 webserver.cc:460] Webserver started at http://127.29.142.190:38995/ using document root <none> and password file <none>
I20260806 06:38:26.469504 18131 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:26.469601 18131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:26.471125 18131 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:38:26.471859 18154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:26.472052 18131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:26.472146 18131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "769bebc24e8149b3a2593701f1feb123"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "81d0027dcf946a8b3e920c8ce060e1dd"
server_key_iv: "d65ec870dc18e158928c1dde37368a66"
server_key_version: "encryptionkey@0"
I20260806 06:38:26.472575 18131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:26.483182 18131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:26.485153 18131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:26.485323 18131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:26.489678 18131 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:33889
I20260806 06:38:26.489755 18223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:33889 every 8 connection(s)
I20260806 06:38:26.490239 18131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:38:26.492084 18224 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:38:26.492795 18224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Bootstrap starting.
I20260806 06:38:26.500208 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 18131
I20260806 06:38:26.500550 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:42053
--local_ip_for_outbound_sockets=127.29.142.129
--tserver_master_addrs=127.29.142.190:33889
--webserver_port=40315
--webserver_interface=127.29.142.129
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:38:26.501509 18224 log.cc:824] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:26.504109 18224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:38:26.504453 18224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Bootstrap complete.
I20260806 06:38:26.507431 18224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } }
I20260806 06:38:26.507745 18224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 769bebc24e8149b3a2593701f1feb123, State: Initialized, Role: FOLLOWER
I20260806 06:38:26.507951 18224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } }
I20260806 06:38:26.508036 18224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:38:26.508098 18224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:38:26.508169 18224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:38:26.509289 18224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } }
I20260806 06:38:26.509418 18224 leader_election.cc:302] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 769bebc24e8149b3a2593701f1feb123; no voters:
I20260806 06:38:26.509649 18224 leader_election.cc:288] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:38:26.509696 18228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:38:26.509805 18228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 2 LEADER]: Becoming Leader. State: Replica: 769bebc24e8149b3a2593701f1feb123, State: Running, Role: LEADER
I20260806 06:38:26.509948 18224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:38:26.509965 18228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } }
I20260806 06:38:26.510380 18229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "769bebc24e8149b3a2593701f1feb123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } } }
I20260806 06:38:26.510483 18229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:26.510794 18230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 769bebc24e8149b3a2593701f1feb123. Latest consensus state: current_term: 2 leader_uuid: "769bebc24e8149b3a2593701f1feb123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } } }
I20260806 06:38:26.510874 18230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:26.510896 18235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:38:26.511824 18235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:38:26.512002 18235 catalog_manager.cc:1259] Loaded cluster ID: 8f935dede7e44853933a08c9aae6d376
I20260806 06:38:26.512225 18235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:38:26.512447 18235 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 06:38:26.515311 18235 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 06:38:26.519449 18235 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:38:26.519709 18235 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Loaded TSK: 0
I20260806 06:38:26.519960 18235 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:38:26.609726 18227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:26.610172 18227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:26.610276 18227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:26.612934 18227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:26.613008 18227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:26.613059 18227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:38:26.615794 18227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:42053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=40315
--tserver_master_addrs=127.29.142.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:26.616183 18227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:26.616513 18227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:26.626497 18264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:26.626497 18262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:26.627691 18227 server_base.cc:1034] running on GCE node
W20260806 06:38:26.627789 18266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:26.628096 18227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:26.628520 18227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:26.629696 18227 hybrid_clock.cc:648] HybridClock initialized: now 1785998306629670 us; error 44 us; skew 500 ppm
Aug 06 06:38:26 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:26.634405 18227 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:38:26.635104 18227 webserver.cc:460] Webserver started at http://127.29.142.129:40315/ using document root <none> and password file <none>
I20260806 06:38:26.635353 18227 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:26.635452 18227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:26.636977 18227 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:38:26.637758 18279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:26.637908 18227 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:38:26.638002 18227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "bc4a424401414d848570034b484651a3"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "e8f9295c2088cec45d0b24c4990be8d4"
server_key_iv: "ca12254674a4f16a3f1404a71d22eace"
server_key_version: "encryptionkey@0"
I20260806 06:38:26.638414 18227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:26.654064 18227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:26.656443 18227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:26.656641 18227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:26.656947 18227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:26.657524 18291 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:38:26.659502 18227 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:38:26.659554 18227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:38:26.659600 18227 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:38:26.660264 18227 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:38:26.660313 18227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:38:26.660356 18291 tablet_bootstrap.cc:492] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Bootstrap starting.
I20260806 06:38:26.668654 18227 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:42053
I20260806 06:38:26.669245 18227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:38:26.669591 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 18227
I20260806 06:38:26.669951 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:35759
--local_ip_for_outbound_sockets=127.29.142.130
--tserver_master_addrs=127.29.142.190:33889
--webserver_port=39301
--webserver_interface=127.29.142.130
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:38:26.673082 18291 log.cc:824] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:26.674397 18439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:42053 every 8 connection(s)
I20260806 06:38:26.676106 18291 tablet_bootstrap.cc:492] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:38:26.676463 18291 tablet_bootstrap.cc:492] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Bootstrap complete.
I20260806 06:38:26.677742 18291 ts_tablet_manager.cc:1397] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.012s
I20260806 06:38:26.678920 18291 raft_consensus.cc:348] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:26.679167 18291 raft_consensus.cc:729] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc4a424401414d848570034b484651a3, State: Initialized, Role: FOLLOWER
I20260806 06:38:26.679324 18291 consensus_queue.cc:260] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:26.679654 18291 ts_tablet_manager.cc:1428] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
Aug 06 06:38:26 dist-test-slave-3kk6 krb5kdc[17370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:26.686722 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.672630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:37715)
0806 06:38:26.673354 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:26.673392 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:26.674032 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:26.675101 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:26.675106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:26.675550 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:26.675836 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.675844 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.677737 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.677739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.679380 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.679384 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.680165 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.680166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.680336 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.680997 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:26.681009 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:26.682814 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:26.684878 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:26.684886 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:26.684888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:26.685141 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:26.685442 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:26.685443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:26.685443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:26.685559 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:26.686014 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:26.686022 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:26.686169 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 06:38:26.686282 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 06:38:26.686472 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":97,"threads_started":1}
I20260806 06:38:26.686722 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.672693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:37715 (local address 127.29.142.190:33889)
0806 06:38:26.674585 (+ 1892us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:26.674596 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:26.674643 (+ 47us) server_negotiation.cc:407] Connection header received
0806 06:38:26.674811 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:26.674818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:26.674886 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:26.675034 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:26.675996 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.677610 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.679539 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.680065 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.680152 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.682962 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:26.682967 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:26.682968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:26.682979 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:26.684774 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:26.685239 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:26.685242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:26.685243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:26.685357 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:26.685631 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:26.685632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:26.685633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:26.685933 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:26.686035 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:26.686342 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 06:38:26.686471 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1828,"thread_start_us":99,"threads_started":1}
I20260806 06:38:26.687425 18440 heartbeater.cc:347] Connected to a master server at 127.29.142.190:33889
I20260806 06:38:26.687548 18440 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:26.687740 18440 heartbeater.cc:511] Master 127.29.142.190:33889 requested a full tablet report, sending...
I20260806 06:38:26.688474 18176 ts_manager.cc:194] Registered new tserver with Master: bc4a424401414d848570034b484651a3 (127.29.142.129:42053)
I20260806 06:38:26.689584 18176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:37715
I20260806 06:38:26.690783 18440 heartbeater.cc:503] Master 127.29.142.190:33889 was elected leader, sending a full tablet report...
I20260806 06:38:26.693799 18450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.689116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42053 (local address 127.0.0.1:55212)
0806 06:38:26.690109 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:26.690148 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:26.690217 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:26.690747 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:26.690750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:26.690752 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:26.690820 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.690823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.691987 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.691988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.692791 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.692793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.693469 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.693470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.693556 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.693570 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:38:26.693661 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 06:38:26.693721 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":956,"thread_start_us":117,"threads_started":1}
I20260806 06:38:26.693799 18451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.689915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55212 (local address 127.29.142.129:42053)
0806 06:38:26.690343 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:26.690347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:26.690356 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:38:26.690387 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:26.690389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:26.690552 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:26.690633 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:26.690910 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.691891 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.692888 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.693371 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.693470 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.693526 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:26.693694 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 06:38:26.693735 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":99,"threads_started":1}
I20260806 06:38:26.694491 18322 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ce1991bdfd468fa07d6f4a9202caec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:26 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:55212
I20260806 06:38:26.695127 18455 tablet_replica.cc:330] stopping tablet replica
I20260806 06:38:26.695258 18455 raft_consensus.cc:2229] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:38:26.695338 18455 raft_consensus.cc:2258] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:38:26.695663 18455 ts_tablet_manager.cc:1905] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:38:26.697826 18455 ts_tablet_manager.cc:1918] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:38:26.697945 18455 log.cc:1196] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal/wals/97ce1991bdfd468fa07d6f4a9202caec
I20260806 06:38:26.698246 18455 ts_tablet_manager.cc:1939] T 97ce1991bdfd468fa07d6f4a9202caec P bc4a424401414d848570034b484651a3: Deleting consensus metadata
I20260806 06:38:26.698887 18158 catalog_manager.cc:4841] TS bc4a424401414d848570034b484651a3 (127.29.142.129:42053): tablet 97ce1991bdfd468fa07d6f4a9202caec (table test-table [id=35b711488789481184da73cfbccc727a]) successfully deleted
W20260806 06:38:26.765362 18443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:26.765597 18443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:26.765623 18443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:26.766844 18443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:26.766896 18443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:26.766923 18443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:38:26.768462 18443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:35759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=39301
--tserver_master_addrs=127.29.142.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:26.768749 18443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:26.769011 18443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:26.774652 18465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:26.774885 18443 server_base.cc:1034] running on GCE node
W20260806 06:38:26.774726 18466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:26.774683 18469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:26.775262 18443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:26.775568 18443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:26.776770 18443 hybrid_clock.cc:648] HybridClock initialized: now 1785998306776744 us; error 43 us; skew 500 ppm
Aug 06 06:38:26 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:26.779774 18443 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:38:26.780273 18443 webserver.cc:460] Webserver started at http://127.29.142.130:39301/ using document root <none> and password file <none>
I20260806 06:38:26.780529 18443 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:26.780628 18443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:26.782109 18443 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:26.782882 18481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:26.783067 18443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:26.783180 18443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "f503992182b24a079f7162ffdd66308d"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "b2adcac1dccbe64ee363f98894989992"
server_key_iv: "7e852272d0d1bc53733b5edeaeac1c4e"
server_key_version: "encryptionkey@0"
I20260806 06:38:26.783578 18443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:26.804229 18443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:26.806511 18443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:26.806744 18443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:26.807024 18443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:26.807498 18490 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:38:26.809002 18443 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:38:26.809087 18443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:38:26.809139 18443 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:38:26.809839 18443 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:38:26.809926 18443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:38:26.810041 18490 tablet_bootstrap.cc:492] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Bootstrap starting.
I20260806 06:38:26.812078 18490 log.cc:824] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:26.815253 18490 tablet_bootstrap.cc:492] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:38:26.815582 18490 tablet_bootstrap.cc:492] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Bootstrap complete.
I20260806 06:38:26.815770 18490 ts_tablet_manager.cc:1397] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260806 06:38:26.817129 18443 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:35759
I20260806 06:38:26.817178 18635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:35759 every 8 connection(s)
I20260806 06:38:26.817500 18443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:38:26.817436 18490 raft_consensus.cc:348] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:26.818044 18490 raft_consensus.cc:729] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f503992182b24a079f7162ffdd66308d, State: Initialized, Role: FOLLOWER
I20260806 06:38:26.818207 18490 consensus_queue.cc:260] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } } peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } }
I20260806 06:38:26.818542 18490 ts_tablet_manager.cc:1428] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260806 06:38:26.818876 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 18443
I20260806 06:38:26.819197 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:34803
--local_ip_for_outbound_sockets=127.29.142.131
--tserver_master_addrs=127.29.142.190:33889
--webserver_port=34663
--webserver_interface=127.29.142.131
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:38:26 dist-test-slave-3kk6 krb5kdc[17370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:26.828562 18639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.818387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:33309)
0806 06:38:26.818566 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:26.818610 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:26.818845 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:26.819123 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:26.819127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:26.819563 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:26.819879 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.819886 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.820610 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.820611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.821520 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.821523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.822055 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.822055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.822157 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.823147 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:26.823156 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:26.824942 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:26.827131 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:26.827139 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:26.827141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:26.827383 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:26.827619 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:26.827620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:26.827620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:26.827724 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:26.828044 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:26.828048 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:26.828170 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 06:38:26.828254 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:38:26.828389 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":109,"threads_started":1}
I20260806 06:38:26.828562 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.818405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:33309 (local address 127.29.142.190:33889)
0806 06:38:26.818464 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:26.818466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:26.818735 (+ 269us) server_negotiation.cc:407] Connection header received
0806 06:38:26.818958 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:26.818959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:26.818977 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:26.819042 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:26.819986 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.820479 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.821619 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.821963 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.822042 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.825078 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:26.825080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:26.825080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:26.825091 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:26.827036 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:26.827468 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:26.827470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:26.827470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:26.827542 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:26.827796 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:26.827797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:26.827797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:26.827969 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:26.828054 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:26.828309 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 06:38:26.828423 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:38:26.829172 18636 heartbeater.cc:347] Connected to a master server at 127.29.142.190:33889
I20260806 06:38:26.829293 18636 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:26.829519 18636 heartbeater.cc:511] Master 127.29.142.190:33889 requested a full tablet report, sending...
I20260806 06:38:26.830186 18176 ts_manager.cc:194] Registered new tserver with Master: f503992182b24a079f7162ffdd66308d (127.29.142.130:35759)
I20260806 06:38:26.831068 18176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:33309
I20260806 06:38:26.832100 18636 heartbeater.cc:503] Master 127.29.142.190:33889 was elected leader, sending a full tablet report...
Aug 06 06:38:26 dist-test-slave-3kk6 krb5kdc[17370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for kudu/127.29.142.130@KRBTEST.COM
I20260806 06:38:26.839021 18450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.830538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:35759 (local address 127.0.0.1:40734)
0806 06:38:26.830713 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:26.830729 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:26.830782 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:26.830970 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:26.830971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:26.831350 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:26.831388 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.831390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.832445 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.832446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.832955 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.832956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.833489 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.833490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.833550 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.834144 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:26.834149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:26.835761 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:26.837852 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:26.837861 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:26.837863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:26.838068 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:26.838314 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:26.838315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:26.838315 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:26.838377 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:26.838743 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:26.838757 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:26.838807 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 06:38:26.838859 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:38:26.838949 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260806 06:38:26.839205 18642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.830606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40734 (local address 127.29.142.130:35759)
0806 06:38:26.830801 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:26.830806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:26.830836 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:38:26.830874 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:26.830875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:26.830887 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:26.830967 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:26.831474 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.832372 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.833045 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.833399 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.833489 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.835859 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:26.835868 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:26.835869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:26.835877 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:26.837754 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:26.838157 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:26.838161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:26.838161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:26.838227 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:26.838440 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:26.838440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:26.838441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:26.838637 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:26.838751 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:26.838926 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 06:38:26.839045 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":135,"threads_started":1}
I20260806 06:38:26.839780 18545 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ce1991bdfd468fa07d6f4a9202caec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:26 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:40734
I20260806 06:38:26.840504 18645 tablet_replica.cc:330] stopping tablet replica
I20260806 06:38:26.840638 18645 raft_consensus.cc:2229] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:38:26.840727 18645 raft_consensus.cc:2258] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:38:26.841127 18645 ts_tablet_manager.cc:1905] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:38:26.843475 18645 ts_tablet_manager.cc:1918] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:38:26.843576 18645 log.cc:1196] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/97ce1991bdfd468fa07d6f4a9202caec
I20260806 06:38:26.843843 18645 ts_tablet_manager.cc:1939] T 97ce1991bdfd468fa07d6f4a9202caec P f503992182b24a079f7162ffdd66308d: Deleting consensus metadata
I20260806 06:38:26.844410 18157 catalog_manager.cc:4841] TS f503992182b24a079f7162ffdd66308d (127.29.142.130:35759): tablet 97ce1991bdfd468fa07d6f4a9202caec (table test-table [id=35b711488789481184da73cfbccc727a]) successfully deleted
W20260806 06:38:26.911468 18640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:26.911720 18640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:26.911753 18640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:26.912953 18640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:26.913002 18640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:26.913031 18640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:38:26.914589 18640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:34803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=34663
--tserver_master_addrs=127.29.142.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:26.914916 18640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:26.915205 18640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:26.920747 18652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:26.920810 18653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:26.920747 18655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:26.920862 18640 server_base.cc:1034] running on GCE node
I20260806 06:38:26.921221 18640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:26.921514 18640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:26.922725 18640 hybrid_clock.cc:648] HybridClock initialized: now 1785998306922706 us; error 32 us; skew 500 ppm
Aug 06 06:38:26 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:26.925536 18640 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:38:26.926025 18640 webserver.cc:460] Webserver started at http://127.29.142.131:34663/ using document root <none> and password file <none>
I20260806 06:38:26.926290 18640 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:26.926393 18640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:26.927940 18640 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:26.928583 18664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:26.928776 18640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:38:26.928903 18640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "4d98aeb86d97444bbeb1ef704adb3816"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "a2a6f682de4d3ccb0981c9c4659451e6"
server_key_iv: "c9c01f73fee855ae1ac6c197b9bc71f7"
server_key_version: "encryptionkey@0"
I20260806 06:38:26.929284 18640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:26.946151 18640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:26.948372 18640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:26.948560 18640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:26.948827 18640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:26.949175 18640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:26.949232 18640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:26.949270 18640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:26.949316 18640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:26.955473 18640 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:34803
I20260806 06:38:26.955574 18823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:34803 every 8 connection(s)
I20260806 06:38:26.955972 18640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:38:26 dist-test-slave-3kk6 krb5kdc[17370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998306, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:26.965251 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.956826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:48907 (local address 127.29.142.190:33889)
0806 06:38:26.956949 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:26.956952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:26.957198 (+ 246us) server_negotiation.cc:407] Connection header received
0806 06:38:26.957706 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:26.957707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:26.957721 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:26.957775 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:26.958810 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.959314 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.960313 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.960514 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.960585 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.962817 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:26.962819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:26.962819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:26.962826 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:26.964069 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:26.964456 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:26.964457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:26.964457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:26.964508 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:26.964725 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:26.964726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:26.964726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:26.964826 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:26.964877 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:26.965117 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 06:38:26.965183 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 06:38:26.965485 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.956816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:48907)
0806 06:38:26.957032 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:26.957089 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:26.957613 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:26.957843 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:26.957847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:26.958444 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:26.958643 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.958702 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.959395 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.959396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.960204 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.960206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.960600 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.960600 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.960710 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.961236 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:26.961244 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:26.962726 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:26.964144 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:26.964151 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:26.964152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:26.964387 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:26.964569 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:26.964569 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:26.964569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:26.964669 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:26.964883 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:26.964888 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:26.965003 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 06:38:26.965090 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:38:26.965221 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":126,"threads_started":1}
I20260806 06:38:26.965867 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 18640
I20260806 06:38:26.966140 18824 heartbeater.cc:347] Connected to a master server at 127.29.142.190:33889
I20260806 06:38:26.966266 18824 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:26.966468 18824 heartbeater.cc:511] Master 127.29.142.190:33889 requested a full tablet report, sending...
I20260806 06:38:26.966940 18176 ts_manager.cc:194] Registered new tserver with Master: 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803)
I20260806 06:38:26.967321 18176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:48907
I20260806 06:38:26.972770 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.966311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50854 (local address 127.29.142.190:33889)
0806 06:38:26.966371 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:26.966373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:26.966558 (+ 185us) server_negotiation.cc:407] Connection header received
0806 06:38:26.966642 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:26.966643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:26.966671 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:26.966728 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:26.967418 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.967889 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.968668 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.968882 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.968957 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.970114 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:26.970115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:26.970115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:26.970123 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:26.971391 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:26.971764 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:26.971765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:26.971766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:26.971827 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:26.972074 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:26.972076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:26.972076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:26.972236 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:26.972411 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:26.972539 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 06:38:26.972656 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:38:26.974331 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:38:26.977370 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.974754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50870 (local address 127.29.142.190:33889)
0806 06:38:26.974797 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:26.974799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:26.974970 (+ 171us) server_negotiation.cc:407] Connection header received
0806 06:38:26.975083 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:26.975084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:26.975155 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:26.975247 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:38:26.975426 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.975919 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.976682 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.976881 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.976934 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.977065 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:38:26.977162 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:38:26.977225 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:26.977257 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 06:38:26.977272 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:38:26.977726 18176 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:50870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:38:26.979524 18176 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:38:26.981740 18545 tablet_service.cc:1467] Processing CreateTablet for tablet b6a230180b7d45a79ac0574b55c5ca7b (DEFAULT_TABLE table=test-table [id=00e2f03d37124232b93fdaf7c42cf636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:38:26.981746 18322 tablet_service.cc:1467] Processing CreateTablet for tablet b6a230180b7d45a79ac0574b55c5ca7b (DEFAULT_TABLE table=test-table [id=00e2f03d37124232b93fdaf7c42cf636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:38:26.981971 18545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6a230180b7d45a79ac0574b55c5ca7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:26.981976 18322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6a230180b7d45a79ac0574b55c5ca7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:26.983639 18490 tablet_bootstrap.cc:492] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d: Bootstrap starting.
I20260806 06:38:26.983723 18291 tablet_bootstrap.cc:492] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3: Bootstrap starting.
I20260806 06:38:26.984053 18291 tablet_bootstrap.cc:654] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:26.984090 18490 tablet_bootstrap.cc:654] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:26.984622 18291 tablet_bootstrap.cc:492] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3: No bootstrap required, opened a new log
I20260806 06:38:26.984671 18291 ts_tablet_manager.cc:1397] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:26.984781 18291 raft_consensus.cc:348] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:26.984872 18291 raft_consensus.cc:374] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:26.984929 18291 raft_consensus.cc:729] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc4a424401414d848570034b484651a3, State: Initialized, Role: FOLLOWER
I20260806 06:38:26.984987 18291 consensus_queue.cc:260] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:26.985090 18291 ts_tablet_manager.cc:1428] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:38:26.985611 18490 tablet_bootstrap.cc:492] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d: No bootstrap required, opened a new log
I20260806 06:38:26.985697 18490 ts_tablet_manager.cc:1397] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260806 06:38:26.985849 18490 raft_consensus.cc:348] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:26.985908 18490 raft_consensus.cc:374] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:26.985944 18490 raft_consensus.cc:729] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f503992182b24a079f7162ffdd66308d, State: Initialized, Role: FOLLOWER
I20260806 06:38:26.986003 18490 consensus_queue.cc:260] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:26.986101 18490 ts_tablet_manager.cc:1428] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:26.986479 18450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.981466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:34803 (local address 127.0.0.1:57094)
0806 06:38:26.981605 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:26.981617 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:26.981692 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:26.982002 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:26.982003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:26.982006 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:26.982053 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.982054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.983443 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.983444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.984549 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:26.984553 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.986221 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.986223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:26.986288 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.986292 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:38:26.986355 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:38:26.986400 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 06:38:26.986637 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:26.981551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57094 (local address 127.29.142.131:34803)
0806 06:38:26.981839 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:26.981844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:26.981854 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:38:26.981894 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:26.981896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:26.981912 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:26.982000 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:26.982138 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.983310 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.984647 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:26.985379 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:26.986221 (+ 842us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:26.986294 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:26.986370 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:38:26.986486 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":114,"threads_started":1}
I20260806 06:38:26.987423 18732 tablet_service.cc:1467] Processing CreateTablet for tablet b6a230180b7d45a79ac0574b55c5ca7b (DEFAULT_TABLE table=test-table [id=00e2f03d37124232b93fdaf7c42cf636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:38:26.987802 18732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6a230180b7d45a79ac0574b55c5ca7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:26.990761 18835 tablet_bootstrap.cc:492] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816: Bootstrap starting.
I20260806 06:38:26.991374 18835 tablet_bootstrap.cc:654] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:26.992004 18835 log.cc:824] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:26.992774 18835 tablet_bootstrap.cc:492] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816: No bootstrap required, opened a new log
I20260806 06:38:26.992831 18835 ts_tablet_manager.cc:1397] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:38:26.994118 18835 raft_consensus.cc:348] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:26.994257 18835 raft_consensus.cc:374] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:26.994282 18835 raft_consensus.cc:729] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d98aeb86d97444bbeb1ef704adb3816, State: Initialized, Role: FOLLOWER
I20260806 06:38:26.994375 18835 consensus_queue.cc:260] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:26.994588 18835 ts_tablet_manager.cc:1428] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:38:26.994683 18824 heartbeater.cc:503] Master 127.29.142.190:33889 was elected leader, sending a full tablet report...
I20260806 06:38:27.022845 18837 raft_consensus.cc:482] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:38:27.022981 18837 raft_consensus.cc:504] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:27.023459 18837 leader_election.cc:288] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f503992182b24a079f7162ffdd66308d (127.29.142.130:35759), bc4a424401414d848570034b484651a3 (127.29.142.129:42053)
I20260806 06:38:27.027593 18451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.023792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:32961 (local address 127.29.142.129:42053)
0806 06:38:27.023840 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:27.023842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:27.023855 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:38:27.023888 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:27.023889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:27.023899 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:27.023987 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:27.024176 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:27.024973 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:27.026534 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:27.027163 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:27.027252 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:27.027264 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:27.027471 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 06:38:27.027515 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:38:27.027608 18642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.023768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:56219 (local address 127.29.142.130:35759)
0806 06:38:27.023826 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:27.023828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:27.023983 (+ 155us) server_negotiation.cc:407] Connection header received
0806 06:38:27.024021 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:27.024023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:27.024051 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:27.024116 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:27.024283 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:27.025020 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:27.026533 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:27.027212 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:27.027281 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:27.027354 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:27.027471 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 06:38:27.027515 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:38:27.027603 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.023686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42053 (local address 127.29.142.131:32961)
0806 06:38:27.023752 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:27.023765 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:27.023814 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:27.023995 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:27.023996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:27.023998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:27.024081 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:27.024082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:27.025078 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:27.025079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:27.026419 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:27.026422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:27.027284 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:27.027285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:27.027380 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:27.027384 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:38:27.027450 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:38:27.027507 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":54}
I20260806 06:38:27.027613 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.023689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:35759 (local address 127.29.142.131:56219)
0806 06:38:27.023902 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:27.023911 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:27.023971 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:27.024140 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:27.024141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:27.024143 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:27.024203 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:27.024204 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:27.025093 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:27.025093 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:27.026419 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:27.026422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:27.027289 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:27.027290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:27.027379 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:27.027384 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:38:27.027450 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:38:27.027507 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":87,"thread_start_us":114,"threads_started":1}
I20260806 06:38:27.027957 18371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6a230180b7d45a79ac0574b55c5ca7b" candidate_uuid: "4d98aeb86d97444bbeb1ef704adb3816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc4a424401414d848570034b484651a3" is_pre_election: true
I20260806 06:38:27.028154 18371 raft_consensus.cc:2454] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d98aeb86d97444bbeb1ef704adb3816 in term 0.
I20260806 06:38:27.028261 18575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6a230180b7d45a79ac0574b55c5ca7b" candidate_uuid: "4d98aeb86d97444bbeb1ef704adb3816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f503992182b24a079f7162ffdd66308d" is_pre_election: true
I20260806 06:38:27.028429 18575 raft_consensus.cc:2454] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d98aeb86d97444bbeb1ef704adb3816 in term 0.
I20260806 06:38:27.028446 18667 leader_election.cc:302] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d98aeb86d97444bbeb1ef704adb3816, bc4a424401414d848570034b484651a3; no voters:
I20260806 06:38:27.028604 18837 raft_consensus.cc:2781] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:38:27.028712 18837 raft_consensus.cc:482] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:38:27.028751 18837 raft_consensus.cc:3037] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:27.029521 18837 raft_consensus.cc:504] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:27.029750 18837 leader_election.cc:288] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [CANDIDATE]: Term 1 election: Requested vote from peers f503992182b24a079f7162ffdd66308d (127.29.142.130:35759), bc4a424401414d848570034b484651a3 (127.29.142.129:42053)
I20260806 06:38:27.029838 18575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6a230180b7d45a79ac0574b55c5ca7b" candidate_uuid: "4d98aeb86d97444bbeb1ef704adb3816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f503992182b24a079f7162ffdd66308d"
I20260806 06:38:27.029942 18575 raft_consensus.cc:3037] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:27.029922 18371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6a230180b7d45a79ac0574b55c5ca7b" candidate_uuid: "4d98aeb86d97444bbeb1ef704adb3816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc4a424401414d848570034b484651a3"
I20260806 06:38:27.030009 18371 raft_consensus.cc:3037] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:27.030786 18575 raft_consensus.cc:2454] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d98aeb86d97444bbeb1ef704adb3816 in term 1.
I20260806 06:38:27.030994 18666 leader_election.cc:302] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d98aeb86d97444bbeb1ef704adb3816, f503992182b24a079f7162ffdd66308d; no voters:
I20260806 06:38:27.031147 18837 raft_consensus.cc:2781] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:38:27.031347 18837 raft_consensus.cc:686] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 1 LEADER]: Becoming Leader. State: Replica: 4d98aeb86d97444bbeb1ef704adb3816, State: Running, Role: LEADER
I20260806 06:38:27.031323 18371 raft_consensus.cc:2454] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d98aeb86d97444bbeb1ef704adb3816 in term 1.
I20260806 06:38:27.031462 18837 consensus_queue.cc:237] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:27.032434 18174 catalog_manager.cc:5495] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131). New cstate: current_term: 1 leader_uuid: "4d98aeb86d97444bbeb1ef704adb3816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } health_report { overall_health: UNKNOWN } } }
I20260806 06:38:27.048199 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.045621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57102 (local address 127.29.142.131:34803)
0806 06:38:27.045694 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:27.045696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:27.045707 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:38:27.045754 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:27.045755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:27.045774 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:27.045854 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:38:27.045994 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:27.046485 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:27.047364 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:27.047570 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:27.047650 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:27.047798 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:38:27.047914 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:38:27.048004 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:27.048051 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 06:38:27.048076 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:38:27.050215 18575 raft_consensus.cc:1261] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d [term 1 FOLLOWER]: Refusing update from remote peer 4d98aeb86d97444bbeb1ef704adb3816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:38:27.050267 18371 raft_consensus.cc:1261] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Refusing update from remote peer 4d98aeb86d97444bbeb1ef704adb3816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:38:27.050619 18837 consensus_queue.cc:1035] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:38:27.050951 18839 consensus_queue.cc:1035] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:38:27.052950 18843 mvcc.cc:205] Tried to move back new op lower bound from 7315449065674743808 to 7315449065601753088. Current Snapshot: MvccSnapshot[applied={T|T < 7315449065674743808}]
I20260806 06:38:27.053741 18847 mvcc.cc:205] Tried to move back new op lower bound from 7315449065674743808 to 7315449065601753088. Current Snapshot: MvccSnapshot[applied={T|T < 7315449065674743808}]
I20260806 06:38:27.053758 18844 mvcc.cc:205] Tried to move back new op lower bound from 7315449065674743808 to 7315449065601753088. Current Snapshot: MvccSnapshot[applied={T|T < 7315449065674743808}]
I20260806 06:38:27.055128 18176 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:50870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:38:27.055248 18176 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:50870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:38:27.055994 18176 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Sending DeleteTablet for 3 replicas of tablet b6a230180b7d45a79ac0574b55c5ca7b
I20260806 06:38:27.056299 18545 tablet_service.cc:1514] Processing DeleteTablet for tablet b6a230180b7d45a79ac0574b55c5ca7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:27 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:40734
I20260806 06:38:27.056346 18732 tablet_service.cc:1514] Processing DeleteTablet for tablet b6a230180b7d45a79ac0574b55c5ca7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:27 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:57094
I20260806 06:38:27.056393 18645 tablet_replica.cc:330] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d: stopping tablet replica
I20260806 06:38:27.056401 18322 tablet_service.cc:1514] Processing DeleteTablet for tablet b6a230180b7d45a79ac0574b55c5ca7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:27 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:55212
I20260806 06:38:27.056447 18645 raft_consensus.cc:2229] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:38:27.056478 18455 tablet_replica.cc:330] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3: stopping tablet replica
I20260806 06:38:27.056495 18645 raft_consensus.cc:2258] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:38:27.056520 18455 raft_consensus.cc:2229] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:38:27.056562 18455 raft_consensus.cc:2258] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:38:27.056787 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 18227
I20260806 06:38:27.056926 18645 ts_tablet_manager.cc:1905] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:38:27.059032 18645 ts_tablet_manager.cc:1918] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:38:27.059098 18645 log.cc:1196] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/b6a230180b7d45a79ac0574b55c5ca7b
I20260806 06:38:27.059367 18645 ts_tablet_manager.cc:1939] T b6a230180b7d45a79ac0574b55c5ca7b P f503992182b24a079f7162ffdd66308d: Deleting consensus metadata
I20260806 06:38:27.059738 18157 catalog_manager.cc:4841] TS f503992182b24a079f7162ffdd66308d (127.29.142.130:35759): tablet b6a230180b7d45a79ac0574b55c5ca7b (table test-table [id=00e2f03d37124232b93fdaf7c42cf636]) successfully deleted
I20260806 06:38:27.059960 18854 tablet_replica.cc:330] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816: stopping tablet replica
I20260806 06:38:27.060118 18854 raft_consensus.cc:2229] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:38:27.060289 18854 raft_consensus.cc:2258] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:38:27.064358 18854 ts_tablet_manager.cc:1905] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:38:27.066367 18854 ts_tablet_manager.cc:1918] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:38:27.066453 18854 log.cc:1196] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/b6a230180b7d45a79ac0574b55c5ca7b
I20260806 06:38:27.066809 18854 ts_tablet_manager.cc:1939] T b6a230180b7d45a79ac0574b55c5ca7b P 4d98aeb86d97444bbeb1ef704adb3816: Deleting consensus metadata
I20260806 06:38:27.067371 18156 catalog_manager.cc:4841] TS 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803): tablet b6a230180b7d45a79ac0574b55c5ca7b (table test-table [id=00e2f03d37124232b93fdaf7c42cf636]) successfully deleted
W20260806 06:38:27.071164 18158 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:42053 (error 108)
I20260806 06:38:27.071574 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 18443
I20260806 06:38:27.071762 18450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.071598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42053 (local address 127.0.0.1:55222)
0806 06:38:27.071675 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:27.071725 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:42053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 06:38:27.078050 18158 catalog_manager.cc:4584] TS bc4a424401414d848570034b484651a3 (127.29.142.129:42053): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6a230180b7d45a79ac0574b55c5ca7b: Network error: Client connection negotiation failed: client connection to 127.29.142.129:42053: connect: Connection refused (error 111)
I20260806 06:38:27.082245 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 18640
I20260806 06:38:27.094265 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 18131
I20260806 06:38:27.103477 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:33889
--webserver_interface=127.29.142.190
--webserver_port=38995
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:33889
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:27.189096 18858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:27.189349 18858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:27.189375 18858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:27.190603 18858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:38:27.190687 18858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:27.190754 18858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:27.190784 18858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:38:27.190814 18858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:38:27.192384 18858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:33889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:33889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:27.192678 18858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:27.192960 18858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:27.198104 18863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:27.198124 18866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:27.198124 18864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:27.198338 18858 server_base.cc:1034] running on GCE node
I20260806 06:38:27.198700 18858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:27.199003 18858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:27.200193 18858 hybrid_clock.cc:648] HybridClock initialized: now 1785998307200178 us; error 35 us; skew 500 ppm
Aug 06 06:38:27 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:27.203011 18858 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:38:27.203540 18858 webserver.cc:460] Webserver started at http://127.29.142.190:38995/ using document root <none> and password file <none>
I20260806 06:38:27.203810 18858 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:27.203897 18858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:27.205324 18858 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:27.206089 18876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:27.206326 18858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:27.206462 18858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "769bebc24e8149b3a2593701f1feb123"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "81d0027dcf946a8b3e920c8ce060e1dd"
server_key_iv: "d65ec870dc18e158928c1dde37368a66"
server_key_version: "encryptionkey@0"
I20260806 06:38:27.206931 18858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:27.229025 18858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:27.231549 18858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:27.231756 18858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:27.236495 18858 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:33889
I20260806 06:38:27.236543 18951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:33889 every 8 connection(s)
I20260806 06:38:27.236960 18858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:38:27.238842 18952 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:38:27.239369 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 18858
I20260806 06:38:27.239719 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:42053
--local_ip_for_outbound_sockets=127.29.142.129
--tserver_master_addrs=127.29.142.190:33889
--webserver_port=40315
--webserver_interface=127.29.142.129
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:38:27.239876 18952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Bootstrap starting.
I20260806 06:38:27.249468 18952 log.cc:824] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:27.252359 18952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:38:27.252737 18952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123: Bootstrap complete.
I20260806 06:38:27.255537 18952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } }
I20260806 06:38:27.255810 18952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 769bebc24e8149b3a2593701f1feb123, State: Initialized, Role: FOLLOWER
I20260806 06:38:27.255990 18952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } }
I20260806 06:38:27.256071 18952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:38:27.256139 18952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:38:27.256196 18952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:38:27.257367 18952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } }
I20260806 06:38:27.257493 18952 leader_election.cc:302] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 769bebc24e8149b3a2593701f1feb123; no voters:
I20260806 06:38:27.257759 18952 leader_election.cc:288] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 06:38:27.257795 18959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 06:38:27.257959 18959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [term 3 LEADER]: Becoming Leader. State: Replica: 769bebc24e8149b3a2593701f1feb123, State: Running, Role: LEADER
I20260806 06:38:27.258086 18952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:38:27.258070 18959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } }
I20260806 06:38:27.258421 18961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 769bebc24e8149b3a2593701f1feb123. Latest consensus state: current_term: 3 leader_uuid: "769bebc24e8149b3a2593701f1feb123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } } }
I20260806 06:38:27.258638 18961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:27.258579 18962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "769bebc24e8149b3a2593701f1feb123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "769bebc24e8149b3a2593701f1feb123" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33889 } } }
I20260806 06:38:27.258947 18962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 769bebc24e8149b3a2593701f1feb123 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:27.259100 18969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:38:27.259919 18969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:38:27.260028 18969 catalog_manager.cc:1259] Loaded cluster ID: 8f935dede7e44853933a08c9aae6d376
I20260806 06:38:27.260063 18969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 06:38:27.260416 18969 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fad07f4e9ad google::LogMessage::Fail() at ??:0
@ 0x7fad07f5188a google::LogMessage::SendToLog() at ??:0
@ 0x7fad07f4e4de google::LogMessage::Flush() at ??:0
@ 0x7fad07f4fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fad0fe6f0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fad0fe788b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fad09e1c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fad09e11954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fad0ac756db start_thread at ??:0
@ 0x7fad06a4c61f clone at ??:0
W20260806 06:38:27.358014 18955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:27.358238 18955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:27.358264 18955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:27.359484 18955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:27.359537 18955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:27.359566 18955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:38:27.361076 18955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:42053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=40315
--tserver_master_addrs=127.29.142.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:27.361315 18955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:27.361574 18955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:27.367166 19008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:27.367167 19013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:27.367419 19009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:27.367614 18955 server_base.cc:1034] running on GCE node
I20260806 06:38:27.367861 18955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:27.368189 18955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:27.369393 18955 hybrid_clock.cc:648] HybridClock initialized: now 1785998307369381 us; error 35 us; skew 500 ppm
Aug 06 06:38:27 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:27.372833 18955 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:38:27.373381 18955 webserver.cc:460] Webserver started at http://127.29.142.129:40315/ using document root <none> and password file <none>
I20260806 06:38:27.373631 18955 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:27.373703 18955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:27.375208 18955 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:38:27.375989 19020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:27.376183 18955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:38:27.376286 18955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "bc4a424401414d848570034b484651a3"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "e8f9295c2088cec45d0b24c4990be8d4"
server_key_iv: "ca12254674a4f16a3f1404a71d22eace"
server_key_version: "encryptionkey@0"
I20260806 06:38:27.376701 18955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:27.392459 18955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:27.395088 18955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:27.395268 18955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:27.395529 18955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:27.396040 19027 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:38:27.398242 18955 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:38:27.398309 18955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:38:27.398357 18955 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:38:27.399125 18955 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:38:27.399185 18955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:38:27.399266 19027 tablet_bootstrap.cc:492] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3: Bootstrap starting.
I20260806 06:38:27.406929 18955 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:42053
I20260806 06:38:27.407382 18955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:38:27.407958 19185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:42053 every 8 connection(s)
I20260806 06:38:27.411279 19027 log.cc:824] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:27.414292 19027 tablet_bootstrap.cc:492] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:38:27.414727 19027 tablet_bootstrap.cc:492] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3: Bootstrap complete.
I20260806 06:38:27.416043 19027 ts_tablet_manager.cc:1397] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.010s
I20260806 06:38:27.416996 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 18955
I20260806 06:38:27.417342 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:35759
--local_ip_for_outbound_sockets=127.29.142.130
--tserver_master_addrs=127.29.142.190:33889
--webserver_port=39301
--webserver_interface=127.29.142.130
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:38:27.417563 19027 raft_consensus.cc:348] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:27.417910 19027 raft_consensus.cc:729] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc4a424401414d848570034b484651a3, State: Initialized, Role: FOLLOWER
I20260806 06:38:27.418061 19027 consensus_queue.cc:260] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:27.418397 19027 ts_tablet_manager.cc:1428] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260806 06:38:27.488467 19188 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:38:27.408157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:54753)
0806 06:38:27.408347 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:27.408395 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:27.408983 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:27.488198 (+ 79215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":115,"threads_started":1}
W20260806 06:38:27.488818 19186 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:33889 (0 consecutive failures): Network error: Failed to ping master at 127.29.142.190:33889: Client connection negotiation failed: client connection to 127.29.142.190:33889: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:38:27.489259 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.489101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:50493)
0806 06:38:27.489178 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:27.489231 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 06:38:27.489722 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.489576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:45887)
0806 06:38:27.489631 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:27.489705 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 06:38:27.489866 19186 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:38:27.507805 19192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:27.508078 19192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:27.508126 19192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:27.509348 19192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:27.509420 19192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:27.509469 19192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:38:27.511099 19192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:35759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=39301
--tserver_master_addrs=127.29.142.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:27.511395 19192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:27.511658 19192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:38:27.517127 19192 server_base.cc:1034] running on GCE node
W20260806 06:38:27.517027 19205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:27.517027 19202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:27.517040 19203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:27.517494 19192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:27.517761 19192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:27.518929 19192 hybrid_clock.cc:648] HybridClock initialized: now 1785998307518905 us; error 48 us; skew 500 ppm
Aug 06 06:38:27 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:27.521652 19192 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:38:27.522183 19192 webserver.cc:460] Webserver started at http://127.29.142.130:39301/ using document root <none> and password file <none>
I20260806 06:38:27.522395 19192 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:27.522440 19192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:27.523890 19192 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:27.524547 19215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:27.524748 19192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:27.524842 19192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "f503992182b24a079f7162ffdd66308d"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "b2adcac1dccbe64ee363f98894989992"
server_key_iv: "7e852272d0d1bc53733b5edeaeac1c4e"
server_key_version: "encryptionkey@0"
I20260806 06:38:27.525206 19192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:27.546557 19192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:27.549400 19192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:27.549568 19192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:27.549867 19192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:27.550235 19192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:27.550294 19192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:27.550340 19192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:27.550367 19192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:27.556921 19192 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:35759
I20260806 06:38:27.556962 19374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:35759 every 8 connection(s)
I20260806 06:38:27.557355 19192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:38:27.558504 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.558068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:55157)
0806 06:38:27.558261 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:27.558359 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":78,"threads_started":1}
W20260806 06:38:27.559062 19375 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:33889 (0 consecutive failures): Network error: Failed to ping master at 127.29.142.190:33889: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
I20260806 06:38:27.559480 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.559345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:52595)
0806 06:38:27.559419 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:27.559456 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 06:38:27.559959 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.559840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:38343)
0806 06:38:27.559907 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:27.559939 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 06:38:27.560135 19375 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:38:27.565301 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 19192
I20260806 06:38:27.565575 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:34803
--local_ip_for_outbound_sockets=127.29.142.131
--tserver_master_addrs=127.29.142.190:33889
--webserver_port=34663
--webserver_interface=127.29.142.131
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:27.650697 19380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:27.650981 19380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:27.651027 19380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:27.652225 19380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:27.652311 19380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:27.652355 19380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:38:27.653868 19380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:34803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=34663
--tserver_master_addrs=127.29.142.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:27.654160 19380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:27.654425 19380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:27.660006 19386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:27.660017 19385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:27.660010 19388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:27.660295 19380 server_base.cc:1034] running on GCE node
I20260806 06:38:27.660529 19380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:27.660845 19380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:27.662086 19380 hybrid_clock.cc:648] HybridClock initialized: now 1785998307662077 us; error 50 us; skew 500 ppm
Aug 06 06:38:27 dist-test-slave-3kk6 krb5kdc[17370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:27.664826 19380 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:38:27.665273 19380 webserver.cc:460] Webserver started at http://127.29.142.131:34663/ using document root <none> and password file <none>
I20260806 06:38:27.665514 19380 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:27.665563 19380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:27.667114 19380 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:38:27.667884 19402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:27.668201 19380 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:27.668313 19380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "4d98aeb86d97444bbeb1ef704adb3816"
format_stamp: "Formatted at 2026-08-06 06:38:25 on dist-test-slave-3kk6"
server_key: "a2a6f682de4d3ccb0981c9c4659451e6"
server_key_iv: "c9c01f73fee855ae1ac6c197b9bc71f7"
server_key_version: "encryptionkey@0"
I20260806 06:38:27.668794 19380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:27.697816 19380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:27.700568 19380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:27.700704 19380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:27.700937 19380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:27.701233 19380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:27.701267 19380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:27.701289 19380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:27.701303 19380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:27.707613 19380 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:34803
I20260806 06:38:27.707716 19545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:34803 every 8 connection(s)
I20260806 06:38:27.708103 19380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:38:27.709549 19548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.708880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:34293)
0806 06:38:27.709088 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:27.709178 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":85,"threads_started":1}
W20260806 06:38:27.709851 19546 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:33889 (0 consecutive failures): Network error: Failed to ping master at 127.29.142.190:33889: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
I20260806 06:38:27.710232 19548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.710105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:55191)
0806 06:38:27.710175 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:27.710209 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 06:38:27.710547 19548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:27.710478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:40399)
0806 06:38:27.710501 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:27.710531 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260806 06:38:27.710711 19546 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:38:27.711822 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 19380
I20260806 06:38:28.490828 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:28.490407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:58333)
0806 06:38:28.490623 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:28.490741 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":92,"threads_started":1}
I20260806 06:38:28.560887 19552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:28.560597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:37731)
0806 06:38:28.560767 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:28.560820 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":69,"threads_started":1}
I20260806 06:38:28.711437 19554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:28.711172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:38337)
0806 06:38:28.711330 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:28.711380 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260806 06:38:28.851222 19555 raft_consensus.cc:482] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:38:28.851399 19555 raft_consensus.cc:504] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:28.851816 19555 leader_election.cc:288] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803), f503992182b24a079f7162ffdd66308d (127.29.142.130:35759)
Aug 06 06:38:28 dist-test-slave-3kk6 krb5kdc[17370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.131@KRBTEST.COM
Aug 06 06:38:28 dist-test-slave-3kk6 krb5kdc[17370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998307, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.130@KRBTEST.COM
I20260806 06:38:28.862620 19558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:28.852213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:43697 (local address 127.29.142.130:35759)
0806 06:38:28.852395 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:28.852403 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:28.852430 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:38:28.852882 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:28.852886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:28.852952 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:28.853123 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:28.853871 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:28.854810 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:28.855849 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:28.856254 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:28.856373 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:28.859228 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:28.859231 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:28.859232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:28.859242 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:28.860976 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:28.861396 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:28.861399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:28.861400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:28.861493 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:28.861737 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:28.861738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:28.861738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:28.861990 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:28.862081 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:28.862362 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 06:38:28.862497 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":54,"threads_started":1}
I20260806 06:38:28.862608 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:28.852053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:35759 (local address 127.29.142.129:43697)
0806 06:38:28.852157 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:28.852172 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:28.852241 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:28.853146 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:28.853149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:28.853537 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:28.853734 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:28.853740 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:28.854906 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:28.854907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:28.855750 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:28.855752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:28.856346 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:28.856347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:28.856461 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:28.857125 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:28.857133 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:28.859096 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:28.861080 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:28.861081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:28.861082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:28.861312 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:28.861574 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:28.861574 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:28.861575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:28.861663 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:28.862073 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:28.862080 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:28.862198 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 06:38:28.862307 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 06:38:28.862509 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":42}
I20260806 06:38:28.862658 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:28.852201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:38503 (local address 127.29.142.131:34803)
0806 06:38:28.852356 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:28.852369 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:28.852408 (+ 39us) server_negotiation.cc:407] Connection header received
0806 06:38:28.852874 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:28.852876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:28.852920 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:28.853041 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:28.853870 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:28.854828 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:28.855844 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:28.856261 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:28.856386 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:28.858911 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:28.858915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:28.858915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:28.858926 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:28.860883 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:28.861359 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:28.861363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:28.861363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:28.861465 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:28.861734 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:28.861735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:28.861736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:28.861989 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:28.862081 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:28.862392 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 06:38:28.862539 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260806 06:38:28.862749 19556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:28.852060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:34803 (local address 127.29.142.129:38503)
0806 06:38:28.852228 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:28.852241 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:28.852302 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:28.853146 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:28.853149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:28.853538 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:28.853734 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:28.853740 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:28.854953 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:28.854954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:28.855748 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:28.855750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:28.856363 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:28.856364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:28.856462 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:28.857183 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:28.857188 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:28.858784 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:28.860994 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:28.861001 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:28.861004 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:28.861267 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:28.861547 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:28.861548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:28.861548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:28.861662 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:28.862075 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:28.862081 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:28.862215 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 06:38:28.862307 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 06:38:28.862593 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":186,"thread_start_us":63,"threads_started":1}
W20260806 06:38:28.863576 19021 leader_election.cc:341] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803): Not found: Tablet not found: b6a230180b7d45a79ac0574b55c5ca7b
W20260806 06:38:28.864104 19022 leader_election.cc:341] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f503992182b24a079f7162ffdd66308d (127.29.142.130:35759): Not found: Tablet not found: b6a230180b7d45a79ac0574b55c5ca7b
I20260806 06:38:28.864151 19022 leader_election.cc:302] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc4a424401414d848570034b484651a3; no voters: 4d98aeb86d97444bbeb1ef704adb3816, f503992182b24a079f7162ffdd66308d
I20260806 06:38:28.864262 19555 raft_consensus.cc:2726] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:38:29.491720 19562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:29.491449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:46477)
0806 06:38:29.491601 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:29.491663 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260806 06:38:29.561936 19563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:29.561640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:32865)
0806 06:38:29.561834 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:29.561887 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":89,"threads_started":1}
I20260806 06:38:29.712327 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:29.712022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:55013)
0806 06:38:29.712198 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:29.712277 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":76,"threads_started":1}
I20260806 06:38:30.492627 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:30.492320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:35077)
0806 06:38:30.492531 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:30.492576 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":111,"threads_started":1}
I20260806 06:38:30.562940 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:30.562581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:50663)
0806 06:38:30.562832 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:30.562893 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":169,"threads_started":1}
I20260806 06:38:30.713253 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:30.712955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:56239)
0806 06:38:30.713151 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:30.713203 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":105,"threads_started":1}
I20260806 06:38:31.037128 19571 raft_consensus.cc:482] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:38:31.037241 19571 raft_consensus.cc:504] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:31.037493 19571 leader_election.cc:288] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803), f503992182b24a079f7162ffdd66308d (127.29.142.130:35759)
W20260806 06:38:31.037973 19022 leader_election.cc:341] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f503992182b24a079f7162ffdd66308d (127.29.142.130:35759): Not found: Tablet not found: b6a230180b7d45a79ac0574b55c5ca7b
W20260806 06:38:31.038064 19021 leader_election.cc:341] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803): Not found: Tablet not found: b6a230180b7d45a79ac0574b55c5ca7b
I20260806 06:38:31.038160 19021 leader_election.cc:302] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc4a424401414d848570034b484651a3; no voters: 4d98aeb86d97444bbeb1ef704adb3816, f503992182b24a079f7162ffdd66308d
I20260806 06:38:31.038252 19571 raft_consensus.cc:2726] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:38:31.493414 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:31.493188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:35063)
0806 06:38:31.493325 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:31.493363 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260806 06:38:31.563794 19573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:31.563508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:42281)
0806 06:38:31.563701 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:31.563746 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":102,"threads_started":1}
I20260806 06:38:31.714198 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:31.713944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:39447)
0806 06:38:31.714092 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:31.714143 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260806 06:38:32.494357 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:32.494093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:36067)
0806 06:38:32.494253 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:32.494306 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260806 06:38:32.564554 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:32.564313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:51201)
0806 06:38:32.564451 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:32.564502 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260806 06:38:32.715225 19578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:32.714951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:43227)
0806 06:38:32.715118 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:32.715173 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":74,"threads_started":1}
I20260806 06:38:33.495390 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:33.495097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:54559)
0806 06:38:33.495281 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:33.495337 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
I20260806 06:38:33.565467 19580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:33.565145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:35119)
0806 06:38:33.565359 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:33.565415 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":85,"threads_started":1}
I20260806 06:38:33.716142 19582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:33.715871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:40765)
0806 06:38:33.716037 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:33.716091 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":78,"threads_started":1}
I20260806 06:38:34.136473 19585 raft_consensus.cc:482] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:38:34.136580 19585 raft_consensus.cc:504] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:34.136804 19585 leader_election.cc:288] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803), f503992182b24a079f7162ffdd66308d (127.29.142.130:35759)
W20260806 06:38:34.137238 19021 leader_election.cc:341] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803): Not found: Tablet not found: b6a230180b7d45a79ac0574b55c5ca7b
W20260806 06:38:34.137320 19022 leader_election.cc:341] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f503992182b24a079f7162ffdd66308d (127.29.142.130:35759): Not found: Tablet not found: b6a230180b7d45a79ac0574b55c5ca7b
I20260806 06:38:34.137388 19022 leader_election.cc:302] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc4a424401414d848570034b484651a3; no voters: 4d98aeb86d97444bbeb1ef704adb3816, f503992182b24a079f7162ffdd66308d
I20260806 06:38:34.137481 19585 raft_consensus.cc:2726] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:38:34.496457 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:34.496123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:57745)
0806 06:38:34.496317 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:34.496387 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":117,"threads_started":1}
I20260806 06:38:34.566427 19589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:34.566123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:33685)
0806 06:38:34.566322 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:34.566376 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":79,"threads_started":1}
I20260806 06:38:34.716970 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:34.716717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:54493)
0806 06:38:34.716869 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:34.716921 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260806 06:38:35.497311 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:35.497032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:40849)
0806 06:38:35.497218 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:35.497261 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":98,"threads_started":1}
I20260806 06:38:35.567258 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:35.566987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:51321)
0806 06:38:35.567155 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:35.567208 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
I20260806 06:38:35.717803 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:35.717529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:52441)
0806 06:38:35.717709 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:35.717746 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":132,"threads_started":1}
I20260806 06:38:36.498252 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:36.497968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:33923)
0806 06:38:36.498141 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:36.498202 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":79,"threads_started":1}
I20260806 06:38:36.568188 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:36.567896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:56929)
0806 06:38:36.568059 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:36.568115 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":79,"threads_started":1}
I20260806 06:38:36.718703 19600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:36.718373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:39033)
0806 06:38:36.718563 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:36.718604 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
I20260806 06:38:37.328715 19604 raft_consensus.cc:482] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:38:37.328896 19604 raft_consensus.cc:504] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:37.329176 19604 leader_election.cc:288] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803), f503992182b24a079f7162ffdd66308d (127.29.142.130:35759)
W20260806 06:38:37.329620 19021 leader_election.cc:341] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803): Not found: Tablet not found: b6a230180b7d45a79ac0574b55c5ca7b
W20260806 06:38:37.329784 19022 leader_election.cc:341] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f503992182b24a079f7162ffdd66308d (127.29.142.130:35759): Not found: Tablet not found: b6a230180b7d45a79ac0574b55c5ca7b
I20260806 06:38:37.329861 19022 leader_election.cc:302] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc4a424401414d848570034b484651a3; no voters: 4d98aeb86d97444bbeb1ef704adb3816, f503992182b24a079f7162ffdd66308d
I20260806 06:38:37.329979 19604 raft_consensus.cc:2726] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:38:37.499166 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:37.498871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:42949)
0806 06:38:37.499054 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:37.499101 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":99,"threads_started":1}
I20260806 06:38:37.569312 19606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:37.568991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:54071)
0806 06:38:37.569196 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:37.569254 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":83,"threads_started":1}
I20260806 06:38:37.719475 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:37.719262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:53613)
0806 06:38:37.719389 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:37.719423 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260806 06:38:38.500106 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:38.499830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:35451)
0806 06:38:38.500010 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:38.500058 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":80,"threads_started":1}
I20260806 06:38:38.570115 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:38.569874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:38533)
0806 06:38:38.570027 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:38.570065 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260806 06:38:38.720263 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:38.719987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:43115)
0806 06:38:38.720159 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:38.720199 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260806 06:38:39.500993 19612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:39.500710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:44307)
0806 06:38:39.500884 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:39.500940 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":79,"threads_started":1}
I20260806 06:38:39.571152 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:39.570871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:51107)
0806 06:38:39.571038 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:39.571095 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260806 06:38:39.721107 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:39.720851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:38625)
0806 06:38:39.721016 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:39.721055 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":85,"threads_started":1}
I20260806 06:38:40.501972 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:40.501695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:52937)
0806 06:38:40.501863 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:40.501917 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260806 06:38:40.572081 19616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:40.571791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:36067)
0806 06:38:40.571967 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:40.572026 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
I20260806 06:38:40.722102 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:40.721803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:53947)
0806 06:38:40.721992 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:40.722051 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":80,"threads_started":1}
I20260806 06:38:41.502949 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:41.502614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:40857)
0806 06:38:41.502866 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:41.502901 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":184,"threads_started":1}
I20260806 06:38:41.572965 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:41.572713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:42953)
0806 06:38:41.572864 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:41.572913 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260806 06:38:41.723166 19622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:41.722889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:38273)
0806 06:38:41.723062 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:41.723112 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260806 06:38:42.038937 19623 raft_consensus.cc:482] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:38:42.039042 19623 raft_consensus.cc:504] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d98aeb86d97444bbeb1ef704adb3816" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 34803 } } peers { permanent_uuid: "f503992182b24a079f7162ffdd66308d" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 35759 } } peers { permanent_uuid: "bc4a424401414d848570034b484651a3" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42053 } }
I20260806 06:38:42.039227 19623 leader_election.cc:288] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803), f503992182b24a079f7162ffdd66308d (127.29.142.130:35759)
W20260806 06:38:42.039620 19022 leader_election.cc:341] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f503992182b24a079f7162ffdd66308d (127.29.142.130:35759): Not found: Tablet not found: b6a230180b7d45a79ac0574b55c5ca7b
W20260806 06:38:42.039700 19021 leader_election.cc:341] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4d98aeb86d97444bbeb1ef704adb3816 (127.29.142.131:34803): Not found: Tablet not found: b6a230180b7d45a79ac0574b55c5ca7b
I20260806 06:38:42.039729 19022 leader_election.cc:302] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc4a424401414d848570034b484651a3; no voters: 4d98aeb86d97444bbeb1ef704adb3816, f503992182b24a079f7162ffdd66308d
I20260806 06:38:42.039803 19623 raft_consensus.cc:2726] T b6a230180b7d45a79ac0574b55c5ca7b P bc4a424401414d848570034b484651a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:38:42.503979 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:42.503674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.129:37609)
0806 06:38:42.503876 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:42.503929 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":102,"threads_started":1}
I20260806 06:38:42.573871 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:42.573603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.130:52361)
0806 06:38:42.573757 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:42.573810 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":71,"threads_started":1}
I20260806 06:38:42.717449 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 18955
I20260806 06:38:42.724877 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:42.724618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:33889 (local address 127.29.142.131:38477)
0806 06:38:42.724769 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:42.724811 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:33889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260806 06:38:42.731696 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 19192
I20260806 06:38:42.743673 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 19380
2026-08-06T06:38:42Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18848 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:38:42 dist-test-slave-3kk6 krb5kdc[19631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:38:42 dist-test-slave-3kk6 krb5kdc[19631](info): set up 2 sockets
Aug 06 06:38:42 dist-test-slave-3kk6 krb5kdc[19631](info): commencing operation
krb5kdc: starting...
W20260806 06:38:43.974047 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:38:44 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:38:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:38:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:38:44.102563 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:43607
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:43607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:44.189828 19646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:44.190083 19646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:44.190110 19646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:44.191372 19646 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:38:44.191428 19646 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:38:44.191442 19646 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:38:44.191454 19646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:38:44.191509 19646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:44.191537 19646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:44.191561 19646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:38:44.191581 19646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:38:44.193186 19646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:43607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:43607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:44.193475 19646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:44.193747 19646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:44.199579 19652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:44.199630 19646 server_base.cc:1034] running on GCE node
W20260806 06:38:44.199568 19657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:44.199568 19654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:44.200058 19646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:44.200404 19646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:44.201561 19646 hybrid_clock.cc:648] HybridClock initialized: now 1785998324201543 us; error 48 us; skew 500 ppm
Aug 06 06:38:44 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998324, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:44.204732 19646 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:38:44.205314 19646 webserver.cc:460] Webserver started at http://127.29.142.190:34741/ using document root <none> and password file <none>
I20260806 06:38:44.205538 19646 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:44.205585 19646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:44.205703 19646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:38:44.207002 19646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "68c0d147b2804ee6a824cdebbe1f7414"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "4e280cf91bbc9ad7582664094f44c3c7"
server_key_iv: "0558091d65fcba87d086dbb6bc5207ff"
server_key_version: "encryptionkey@0"
I20260806 06:38:44.207370 19646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "68c0d147b2804ee6a824cdebbe1f7414"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "4e280cf91bbc9ad7582664094f44c3c7"
server_key_iv: "0558091d65fcba87d086dbb6bc5207ff"
server_key_version: "encryptionkey@0"
I20260806 06:38:44.208914 19646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:38:44.209762 19667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:44.209977 19646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:44.210112 19646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "68c0d147b2804ee6a824cdebbe1f7414"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "4e280cf91bbc9ad7582664094f44c3c7"
server_key_iv: "0558091d65fcba87d086dbb6bc5207ff"
server_key_version: "encryptionkey@0"
I20260806 06:38:44.210211 19646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:44.227774 19646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:44.230309 19646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:44.230505 19646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:44.235139 19646 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:43607
I20260806 06:38:44.235184 19732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:43607 every 8 connection(s)
I20260806 06:38:44.235574 19646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:38:44.236519 19733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:44.238983 19733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Bootstrap starting.
I20260806 06:38:44.239629 19733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:44.239961 19733 log.cc:824] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:44.239970 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 19646
I20260806 06:38:44.240092 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:38:44.240342 30266 external_mini_cluster.cc:1428] Setting key 640226d33196b0fd720c4e23656ee9ed
I20260806 06:38:44.240882 19733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: No bootstrap required, opened a new log
I20260806 06:38:44.243266 19733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } }
I20260806 06:38:44.243441 19733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:44.243511 19733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c0d147b2804ee6a824cdebbe1f7414, State: Initialized, Role: FOLLOWER
I20260806 06:38:44.243693 19733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } }
I20260806 06:38:44.243793 19733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:38:44.243840 19733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:38:44.243891 19733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:44.244488 19733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } }
I20260806 06:38:44.244614 19733 leader_election.cc:302] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68c0d147b2804ee6a824cdebbe1f7414; no voters:
I20260806 06:38:44.244829 19733 leader_election.cc:288] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:38:44.244884 19741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:38:44.245056 19741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 1 LEADER]: Becoming Leader. State: Replica: 68c0d147b2804ee6a824cdebbe1f7414, State: Running, Role: LEADER
I20260806 06:38:44.245129 19733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:38:44.245211 19741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } }
I20260806 06:38:44.245612 19742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68c0d147b2804ee6a824cdebbe1f7414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } } }
I20260806 06:38:44.245692 19742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:44.245959 19743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68c0d147b2804ee6a824cdebbe1f7414. Latest consensus state: current_term: 1 leader_uuid: "68c0d147b2804ee6a824cdebbe1f7414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } } }
I20260806 06:38:44.246255 19743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: This master's current role is: LEADER
I20260806 06:38:44.246464 19751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:38:44.247159 19751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:38:44 dist-test-slave-3kk6 krb5kdc[19631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:44.249347 19751 catalog_manager.cc:1347] Generated new cluster ID: fdfb8ca979f3411289b2f1eb9d6b39cd
I20260806 06:38:44.249403 19751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:38:44.253847 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.240986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39332 (local address 127.29.142.190:43607)
0806 06:38:44.241182 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:44.241190 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:44.241217 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:38:44.242143 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:44.242151 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:44.242204 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:44.242328 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:44.243215 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.244621 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.245790 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.246451 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.246545 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.249782 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:44.249787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:44.249787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:44.249800 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:44.252079 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:44.252496 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:44.252501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:44.252501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:44.252632 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:44.252844 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:44.252845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:44.252845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:44.253173 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:44.253356 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:44.253449 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 06:38:44.253629 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260806 06:38:44.278100 19751 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:38:44.278966 19751 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:38:44.285722 19751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Generated new TSK 0
I20260806 06:38:44.285986 19751 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:38:44.317873 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:43607
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:44.405051 19772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:44.405275 19772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:44.405300 19772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:44.406474 19772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:44.406529 19772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:44.406589 19772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:38:44.408166 19772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:44.408443 19772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:44.408731 19772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:44.414258 19783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:44.414263 19780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:44.414309 19772 server_base.cc:1034] running on GCE node
W20260806 06:38:44.414269 19779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:44.414798 19772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:44.415127 19772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:44.416303 19772 hybrid_clock.cc:648] HybridClock initialized: now 1785998324416298 us; error 41 us; skew 500 ppm
Aug 06 06:38:44 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998324, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:44.419269 19772 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:38:44.419757 19772 webserver.cc:460] Webserver started at http://127.29.142.129:36393/ using document root <none> and password file <none>
I20260806 06:38:44.420027 19772 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:44.420099 19772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:44.420256 19772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:38:44.421386 19772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "ee6b01b560b144b0a5d1f576796184db"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "01f5139f8501a7a578173c4433c104a5"
server_key_iv: "1740bf3150849641c0c84924db62d7cd"
server_key_version: "encryptionkey@0"
I20260806 06:38:44.421763 19772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "ee6b01b560b144b0a5d1f576796184db"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "01f5139f8501a7a578173c4433c104a5"
server_key_iv: "1740bf3150849641c0c84924db62d7cd"
server_key_version: "encryptionkey@0"
I20260806 06:38:44.423276 19772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:38:44.424046 19793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:44.424224 19772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:38:44.424346 19772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "ee6b01b560b144b0a5d1f576796184db"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "01f5139f8501a7a578173c4433c104a5"
server_key_iv: "1740bf3150849641c0c84924db62d7cd"
server_key_version: "encryptionkey@0"
I20260806 06:38:44.424428 19772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:44.455008 19772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:44.457160 19772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:44.457360 19772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:44.457669 19772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:44.458048 19772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:44.458108 19772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:44.458146 19772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:44.458178 19772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:44.464427 19772 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:42987
I20260806 06:38:44.464475 19949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:42987 every 8 connection(s)
I20260806 06:38:44.464843 19772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:38:44 dist-test-slave-3kk6 krb5kdc[19631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998324, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:44.474376 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 19772
I20260806 06:38:44.474500 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:38:44.474785 30266 external_mini_cluster.cc:1428] Setting key 2bdf39b5af2b8d8f523d166e19eb2e8f
I20260806 06:38:44.475526 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.465658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:41171 (local address 127.29.142.190:43607)
0806 06:38:44.465733 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:44.465735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:44.465936 (+ 201us) server_negotiation.cc:407] Connection header received
0806 06:38:44.466440 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:44.466441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:44.466451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:44.466522 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:44.467597 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.468138 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.469111 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.469305 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.469359 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.471690 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:44.471692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:44.471692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:44.471708 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:44.473292 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:44.473952 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:44.473954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:44.473954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:44.474026 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:44.474376 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:44.474377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:44.474378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:44.474623 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:44.474744 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:44.475218 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 06:38:44.475429 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:38:44.475637 19953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.465608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:41171)
0806 06:38:44.465787 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:44.465841 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:44.466351 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:44.466607 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:44.466611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:44.467295 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:44.467502 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.467507 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.468217 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.468218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.469028 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.469030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.469388 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.469389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.469507 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.470029 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:44.470038 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:44.471565 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:44.473421 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:44.473436 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:44.473439 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:44.473862 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:44.474122 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:44.474123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:44.474123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:44.474276 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:44.474779 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:44.474787 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:44.474991 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 06:38:44.475185 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 06:38:44.475375 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260806 06:38:44.476164 19950 heartbeater.cc:347] Connected to a master server at 127.29.142.190:43607
I20260806 06:38:44.476274 19950 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:44.476511 19950 heartbeater.cc:511] Master 127.29.142.190:43607 requested a full tablet report, sending...
I20260806 06:38:44.477137 19685 ts_manager.cc:194] Registered new tserver with Master: ee6b01b560b144b0a5d1f576796184db (127.29.142.129:42987)
I20260806 06:38:44.477933 19685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:41171
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:38:44.509860 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:43607
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:44.595361 19956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:44.595619 19956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:44.595645 19956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:44.596872 19956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:44.596920 19956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:44.596948 19956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:38:44.598484 19956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:44.598780 19956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:44.599072 19956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:38:44.604341 19956 server_base.cc:1034] running on GCE node
W20260806 06:38:44.604324 19966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:44.604358 19963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:44.604315 19964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:44.604789 19956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:44.605010 19956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:44.606217 19956 hybrid_clock.cc:648] HybridClock initialized: now 1785998324606203 us; error 33 us; skew 500 ppm
Aug 06 06:38:44 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998324, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:44.608888 19956 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:38:44.609315 19956 webserver.cc:460] Webserver started at http://127.29.142.130:34537/ using document root <none> and password file <none>
I20260806 06:38:44.609520 19956 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:44.609563 19956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:44.609660 19956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:38:44.610817 19956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "b9ec3df106514e309c5d903cc1536618"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "6f951ecb8420a8294127d158e48256a2"
server_key_iv: "66630582ca242df199370a8ffd45ce63"
server_key_version: "encryptionkey@0"
I20260806 06:38:44.611172 19956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "b9ec3df106514e309c5d903cc1536618"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "6f951ecb8420a8294127d158e48256a2"
server_key_iv: "66630582ca242df199370a8ffd45ce63"
server_key_version: "encryptionkey@0"
I20260806 06:38:44.612505 19956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:38:44.613241 19978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:44.613443 19956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:38:44.613585 19956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "b9ec3df106514e309c5d903cc1536618"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "6f951ecb8420a8294127d158e48256a2"
server_key_iv: "66630582ca242df199370a8ffd45ce63"
server_key_version: "encryptionkey@0"
I20260806 06:38:44.613693 19956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:44.623178 19956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:44.625185 19956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:44.625310 19956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:44.625516 19956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:44.625847 19956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:44.625883 19956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:44.625908 19956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:44.625959 19956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:44.632283 19956 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:34525
I20260806 06:38:44.632351 20133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:34525 every 8 connection(s)
I20260806 06:38:44.632711 19956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:38:44.635823 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 19956
I20260806 06:38:44.635957 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:38:44.636175 30266 external_mini_cluster.cc:1428] Setting key 45bf34e1ae0a82036b0dfb72cea87c88
Aug 06 06:38:44 dist-test-slave-3kk6 krb5kdc[19631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998324, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:44.643754 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.633510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:55753 (local address 127.29.142.190:43607)
0806 06:38:44.633592 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:44.633595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:44.633759 (+ 164us) server_negotiation.cc:407] Connection header received
0806 06:38:44.634246 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:44.634247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:44.634258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:44.634313 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:44.635383 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.636103 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.637528 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.637837 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.637921 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.640702 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:44.640704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:44.640704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:44.640715 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:44.642214 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:44.642691 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:44.642693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:44.642693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:44.642759 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:44.643036 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:44.643037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:44.643037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:44.643165 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:44.643245 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:44.643564 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 06:38:44.643655 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:38:44.643978 20137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.633444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:55753)
0806 06:38:44.633616 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:44.633658 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:44.634160 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:44.634372 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:44.634375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:44.635072 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:44.635291 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.635296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.636210 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.636212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.637438 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.637441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.637972 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.637973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.638095 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.638880 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:44.638893 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:44.640588 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:44.642327 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:44.642341 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:44.642343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:44.642588 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:44.642842 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:44.642843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:44.642843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:44.642964 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:44.643251 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:44.643259 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:44.643420 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 06:38:44.643540 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 06:38:44.643715 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
I20260806 06:38:44.644723 20134 heartbeater.cc:347] Connected to a master server at 127.29.142.190:43607
I20260806 06:38:44.644870 20134 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:44.645102 20134 heartbeater.cc:511] Master 127.29.142.190:43607 requested a full tablet report, sending...
I20260806 06:38:44.645623 19685 ts_manager.cc:194] Registered new tserver with Master: b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525)
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
I20260806 06:38:44.646111 19685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:55753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:38:44.665145 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:43607
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:38:44.752352 20141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:38:44.752594 20141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:38:44.752619 20141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:38:44.753962 20141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:38:44.754037 20141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:38:44.754105 20141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:38:44.755721 20141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:38:44.756006 20141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:38:44.756264 20141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:38:44.761592 20150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:44.761564 20151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:38:44.761564 20153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:38:44.761637 20141 server_base.cc:1034] running on GCE node
I20260806 06:38:44.762039 20141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:38:44.762321 20141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:38:44.763469 20141 hybrid_clock.cc:648] HybridClock initialized: now 1785998324763448 us; error 41 us; skew 500 ppm
Aug 06 06:38:44 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998324, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:38:44.766163 20141 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:38:44.766763 20141 webserver.cc:460] Webserver started at http://127.29.142.131:45131/ using document root <none> and password file <none>
I20260806 06:38:44.767009 20141 fs_manager.cc:359] Metadata directory not provided
I20260806 06:38:44.767081 20141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:38:44.767251 20141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:38:44.768415 20141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "63b22c163965490b85a29f556da28967"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "5e3071932f76c03300e8cf5b16127aff"
server_key_iv: "46ace4dee7c54a8c47c8d4a03b8ff094"
server_key_version: "encryptionkey@0"
I20260806 06:38:44.768836 20141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "63b22c163965490b85a29f556da28967"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "5e3071932f76c03300e8cf5b16127aff"
server_key_iv: "46ace4dee7c54a8c47c8d4a03b8ff094"
server_key_version: "encryptionkey@0"
I20260806 06:38:44.770303 20141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:38:44.771216 20160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:44.771404 20141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:38:44.771521 20141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "63b22c163965490b85a29f556da28967"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "5e3071932f76c03300e8cf5b16127aff"
server_key_iv: "46ace4dee7c54a8c47c8d4a03b8ff094"
server_key_version: "encryptionkey@0"
I20260806 06:38:44.771610 20141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:38:44.791188 20141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:38:44.793265 20141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:38:44.793454 20141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:38:44.793713 20141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:38:44.794052 20141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:38:44.794116 20141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:44.794162 20141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:38:44.794194 20141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:38:44.800462 20141 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:45201
I20260806 06:38:44.800508 20313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:45201 every 8 connection(s)
I20260806 06:38:44.800884 20141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:38:44.801384 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 20141
I20260806 06:38:44.801498 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:38:44.801713 30266 external_mini_cluster.cc:1428] Setting key 741a5bb9055cea192ac2e5713c3850d5
Aug 06 06:38:44 dist-test-slave-3kk6 krb5kdc[19631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998324, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:38:44.810889 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.801718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:37291 (local address 127.29.142.190:43607)
0806 06:38:44.801767 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:44.801769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:44.802054 (+ 285us) server_negotiation.cc:407] Connection header received
0806 06:38:44.802721 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:44.802722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:44.802735 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:44.802776 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:44.803837 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.804347 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.805341 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.805527 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.805582 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.808006 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:44.808008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:44.808008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:44.808016 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:44.809598 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:44.809989 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:44.809990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:44.809991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:44.810048 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:44.810282 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:44.810282 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:44.810283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:44.810392 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:44.810447 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:44.810739 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 06:38:44.810807 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:38:44.811172 20316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.801717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:37291)
0806 06:38:44.801930 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:44.801976 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:44.802566 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:44.802886 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:44.802890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:44.803521 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:38:44.803738 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.803744 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.804442 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.804443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.805249 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.805251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.805611 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.805612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.805720 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.806323 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:38:44.806331 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:38:44.807909 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:38:44.809684 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:44.809690 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:44.809692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:44.809922 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:44.810112 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:38:44.810113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:38:44.810113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:38:44.810224 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:38:44.810451 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:38:44.810456 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:38:44.810577 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 06:38:44.810717 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 06:38:44.810882 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":107,"threads_started":1}
I20260806 06:38:44.811745 20314 heartbeater.cc:347] Connected to a master server at 127.29.142.190:43607
I20260806 06:38:44.811877 20314 heartbeater.cc:464] Registering TS with master...
I20260806 06:38:44.812083 20314 heartbeater.cc:511] Master 127.29.142.190:43607 requested a full tablet report, sending...
I20260806 06:38:44.812496 19685 ts_manager.cc:194] Registered new tserver with Master: 63b22c163965490b85a29f556da28967 (127.29.142.131:45201)
I20260806 06:38:44.812883 19685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:37291
I20260806 06:38:44.813779 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:38:44.821079 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.815182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39344 (local address 127.29.142.190:43607)
0806 06:38:44.815219 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:44.815221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:44.815363 (+ 142us) server_negotiation.cc:407] Connection header received
0806 06:38:44.815407 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:44.815408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:44.815416 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:44.815464 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:38:44.815977 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.816440 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.817212 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.817391 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.817445 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.818595 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:38:44.818596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:38:44.818597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:38:44.818603 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:38:44.820094 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:44.820428 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:44.820429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:44.820430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:44.820486 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:38:44.820645 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:38:44.820646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:38:44.820646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:38:44.820777 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:38:44.820857 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:44.820939 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 06:38:44.821004 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:38:44.822909 19685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:38:44.824517 19685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:38:44.831446 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.827105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:45201 (local address 127.0.0.1:49882)
0806 06:38:44.827251 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:44.827294 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:44.827368 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:44.827935 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:44.827938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:44.827940 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:44.828016 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.828018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.829026 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.829027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.830258 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.830260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.831124 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.831125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.831217 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.831231 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:38:44.831306 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:38:44.831362 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":106,"threads_started":1}
I20260806 06:38:44.831465 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.827290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49882 (local address 127.29.142.131:45201)
0806 06:38:44.827770 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:44.827775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:44.827786 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:38:44.827826 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:44.827828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:44.827850 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:44.827931 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:44.828095 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.828918 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.830367 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.831031 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.831127 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.831187 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:44.831337 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 06:38:44.831379 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":97,"threads_started":1}
I20260806 06:38:44.831619 20336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.827283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:34525 (local address 127.0.0.1:46636)
0806 06:38:44.827516 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:44.827526 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:44.827599 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:44.828206 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:44.828207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:44.828209 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:44.828258 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.828259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.829448 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.829449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.830378 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.830381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.831358 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.831358 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.831441 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.831460 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 06:38:44.831494 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 06:38:44.831538 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260806 06:38:44.831832 20334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.827066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42987 (local address 127.0.0.1:55878)
0806 06:38:44.827238 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:44.827292 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:44.827361 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:44.827938 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:44.827939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:44.827942 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:44.828028 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.828030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.829395 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.829397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.830466 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.830468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.831356 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.831358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.831441 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.831461 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 06:38:44.831506 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:38:44.831538 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":68,"thread_start_us":75,"threads_started":1}
I20260806 06:38:44.832212 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.827498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55878 (local address 127.29.142.129:42987)
0806 06:38:44.827731 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:44.827738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:44.827750 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:38:44.827808 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:44.827811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:44.827828 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:44.828117 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:44.828193 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.829282 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.830549 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.831252 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.831981 (+ 729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.832047 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:44.832073 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:38:44.832117 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":110,"threads_started":1}
I20260806 06:38:44.832360 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.827502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46636 (local address 127.29.142.130:34525)
0806 06:38:44.828044 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:44.828049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:44.828060 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:38:44.828098 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:44.828100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:44.828115 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:44.828194 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:44.828321 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.829366 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.830477 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.831229 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.831811 (+ 582us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.831881 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:44.831910 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 06:38:44.832273 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":99,"threads_started":1}
I20260806 06:38:44.832387 20229 tablet_service.cc:1467] Processing CreateTablet for tablet b210d95771c748d4b6899787b15e066e (DEFAULT_TABLE table=test-table [id=fe6a2181923e4c5a92d42e82acc64afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:38:44.832810 20229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b210d95771c748d4b6899787b15e066e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:44.832891 19858 tablet_service.cc:1467] Processing CreateTablet for tablet b210d95771c748d4b6899787b15e066e (DEFAULT_TABLE table=test-table [id=fe6a2181923e4c5a92d42e82acc64afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:38:44.832942 20044 tablet_service.cc:1467] Processing CreateTablet for tablet b210d95771c748d4b6899787b15e066e (DEFAULT_TABLE table=test-table [id=fe6a2181923e4c5a92d42e82acc64afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:38:44.833245 20044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b210d95771c748d4b6899787b15e066e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:44.833348 19858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b210d95771c748d4b6899787b15e066e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:38:44.835774 20341 tablet_bootstrap.cc:492] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967: Bootstrap starting.
I20260806 06:38:44.836385 20341 tablet_bootstrap.cc:654] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:44.836694 20342 tablet_bootstrap.cc:492] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Bootstrap starting.
I20260806 06:38:44.836999 20341 log.cc:824] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:44.837423 20342 tablet_bootstrap.cc:654] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:44.838002 20341 tablet_bootstrap.cc:492] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967: No bootstrap required, opened a new log
I20260806 06:38:44.838061 20342 log.cc:824] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:44.838086 20341 ts_tablet_manager.cc:1397] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:38:44.838447 20343 tablet_bootstrap.cc:492] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618: Bootstrap starting.
I20260806 06:38:44.838894 20342 tablet_bootstrap.cc:492] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: No bootstrap required, opened a new log
I20260806 06:38:44.838981 20342 ts_tablet_manager.cc:1397] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:38:44.839141 20343 tablet_bootstrap.cc:654] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618: Neither blocks nor log segments found. Creating new log.
I20260806 06:38:44.839751 20343 log.cc:824] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618: Log is configured to *not* fsync() on all Append() calls
I20260806 06:38:44.840366 20341 raft_consensus.cc:348] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } } peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } }
I20260806 06:38:44.840588 20341 raft_consensus.cc:374] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:44.840611 20343 tablet_bootstrap.cc:492] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618: No bootstrap required, opened a new log
I20260806 06:38:44.840639 20341 raft_consensus.cc:729] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b22c163965490b85a29f556da28967, State: Initialized, Role: FOLLOWER
I20260806 06:38:44.840680 20343 ts_tablet_manager.cc:1397] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:38:44.840782 20341 consensus_queue.cc:260] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } } peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } }
I20260806 06:38:44.841120 20314 heartbeater.cc:503] Master 127.29.142.190:43607 was elected leader, sending a full tablet report...
I20260806 06:38:44.841116 20341 ts_tablet_manager.cc:1428] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:38:44.841346 20342 raft_consensus.cc:348] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } } peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } }
I20260806 06:38:44.841526 20342 raft_consensus.cc:374] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:44.841574 20342 raft_consensus.cc:729] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee6b01b560b144b0a5d1f576796184db, State: Initialized, Role: FOLLOWER
I20260806 06:38:44.841693 20342 consensus_queue.cc:260] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } } peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } }
I20260806 06:38:44.841997 20342 ts_tablet_manager.cc:1428] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:38:44.842082 19950 heartbeater.cc:503] Master 127.29.142.190:43607 was elected leader, sending a full tablet report...
I20260806 06:38:44.842815 20343 raft_consensus.cc:348] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } } peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } }
I20260806 06:38:44.842972 20343 raft_consensus.cc:374] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:38:44.843017 20343 raft_consensus.cc:729] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ec3df106514e309c5d903cc1536618, State: Initialized, Role: FOLLOWER
I20260806 06:38:44.843130 20343 consensus_queue.cc:260] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } } peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } }
I20260806 06:38:44.843411 20343 ts_tablet_manager.cc:1428] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:38:44.843472 20134 heartbeater.cc:503] Master 127.29.142.190:43607 was elected leader, sending a full tablet report...
W20260806 06:38:44.883421 20136 tablet.cc:2369] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:38:44.965658 19952 tablet.cc:2369] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:38:44.991881 20348 raft_consensus.cc:482] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:38:44.992036 20348 raft_consensus.cc:504] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } } peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } }
I20260806 06:38:44.992401 20348 leader_election.cc:288] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525), 63b22c163965490b85a29f556da28967 (127.29.142.131:45201)
I20260806 06:38:44.995651 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.992791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:34349 (local address 127.29.142.131:45201)
0806 06:38:44.992850 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:44.992852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:44.992864 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:38:44.992944 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:44.992945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:44.992956 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:44.993039 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:44.993266 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.993719 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.994866 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.995271 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.995332 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.995340 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:44.995527 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 06:38:44.995579 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:38:44.995661 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.992635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:45201 (local address 127.29.142.129:34349)
0806 06:38:44.992795 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:44.992809 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:44.992876 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:44.993043 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:44.993044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:44.993047 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:44.993156 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.993159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.993812 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.993813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.994760 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.994763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.995357 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.995358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.995436 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.995443 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:38:44.995495 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:38:44.995529 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":53,"thread_start_us":104,"threads_started":1}
I20260806 06:38:44.995730 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.992730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:53923 (local address 127.29.142.130:34525)
0806 06:38:44.992790 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:44.992793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:44.992879 (+ 86us) server_negotiation.cc:407] Connection header received
0806 06:38:44.992947 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:44.992948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:44.992958 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:44.993039 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:38:44.993226 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.993718 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.994906 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.995293 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.995390 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.995399 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:44.995606 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 06:38:44.995650 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:38:44.995779 20351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:44.992637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:34525 (local address 127.29.142.129:53923)
0806 06:38:44.992801 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:44.992812 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:38:44.992884 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:38:44.993045 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:38:44.993045 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:38:44.993051 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:38:44.993144 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.993146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.993814 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.993815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.994786 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:38:44.994789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:38:44.995410 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:38:44.995412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:38:44.995518 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:44.995523 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:38:44.995582 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:38:44.995621 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":29,"thread_start_us":80,"threads_started":1}
I20260806 06:38:44.996049 20262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b210d95771c748d4b6899787b15e066e" candidate_uuid: "ee6b01b560b144b0a5d1f576796184db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b22c163965490b85a29f556da28967" is_pre_election: true
I20260806 06:38:44.996222 20262 raft_consensus.cc:2454] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee6b01b560b144b0a5d1f576796184db in term 0.
I20260806 06:38:44.996510 19798 leader_election.cc:302] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b22c163965490b85a29f556da28967, ee6b01b560b144b0a5d1f576796184db; no voters:
I20260806 06:38:44.996546 20069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b210d95771c748d4b6899787b15e066e" candidate_uuid: "ee6b01b560b144b0a5d1f576796184db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ec3df106514e309c5d903cc1536618" is_pre_election: true
I20260806 06:38:44.996685 20069 raft_consensus.cc:2454] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee6b01b560b144b0a5d1f576796184db in term 0.
I20260806 06:38:44.996687 20348 raft_consensus.cc:2781] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:38:44.996754 20348 raft_consensus.cc:482] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:38:44.996790 20348 raft_consensus.cc:3037] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:44.997608 20348 raft_consensus.cc:504] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } } peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } }
I20260806 06:38:44.997807 20348 leader_election.cc:288] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 1 election: Requested vote from peers b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525), 63b22c163965490b85a29f556da28967 (127.29.142.131:45201)
I20260806 06:38:44.997962 20069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b210d95771c748d4b6899787b15e066e" candidate_uuid: "ee6b01b560b144b0a5d1f576796184db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ec3df106514e309c5d903cc1536618"
I20260806 06:38:44.997997 20262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b210d95771c748d4b6899787b15e066e" candidate_uuid: "ee6b01b560b144b0a5d1f576796184db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63b22c163965490b85a29f556da28967"
I20260806 06:38:44.998067 20069 raft_consensus.cc:3037] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:44.998095 20262 raft_consensus.cc:3037] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:38:44.998992 20262 raft_consensus.cc:2454] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee6b01b560b144b0a5d1f576796184db in term 1.
I20260806 06:38:44.999004 20069 raft_consensus.cc:2454] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee6b01b560b144b0a5d1f576796184db in term 1.
I20260806 06:38:44.999182 19798 leader_election.cc:302] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b22c163965490b85a29f556da28967, ee6b01b560b144b0a5d1f576796184db; no voters:
I20260806 06:38:44.999290 20348 raft_consensus.cc:2781] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:38:44.999467 20348 raft_consensus.cc:686] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 1 LEADER]: Becoming Leader. State: Replica: ee6b01b560b144b0a5d1f576796184db, State: Running, Role: LEADER
I20260806 06:38:44.999545 20348 consensus_queue.cc:237] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } } peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } }
I20260806 06:38:45.000483 19686 catalog_manager.cc:5495] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db reported cstate change: term changed from 0 to 1, leader changed from <none> to ee6b01b560b144b0a5d1f576796184db (127.29.142.129). New cstate: current_term: 1 leader_uuid: "ee6b01b560b144b0a5d1f576796184db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } health_report { overall_health: UNKNOWN } } }
I20260806 06:38:45.016358 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:45.013750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55892 (local address 127.29.142.129:42987)
0806 06:38:45.013837 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:38:45.013839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:38:45.013850 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:38:45.013889 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:38:45.013891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:38:45.013909 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:38:45.013973 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:38:45.014092 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:45.014729 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:45.015571 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:38:45.015756 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:38:45.015812 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:38:45.015979 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:38:45.016086 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:38:45.016153 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:38:45.016205 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:38:45.016232 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 06:38:45.018373 20069 raft_consensus.cc:1261] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618 [term 1 FOLLOWER]: Refusing update from remote peer ee6b01b560b144b0a5d1f576796184db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:38:45.018373 20262 raft_consensus.cc:1261] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967 [term 1 FOLLOWER]: Refusing update from remote peer ee6b01b560b144b0a5d1f576796184db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:38:45.018841 20348 consensus_queue.cc:1035] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:38:45.019018 20355 consensus_queue.cc:1035] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [LEADER]: Connected to new peer: Peer: permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:38:45.021693 20358 mvcc.cc:205] Tried to move back new op lower bound from 7315449139272470528 to 7315449139198738432. Current Snapshot: MvccSnapshot[applied={T|T < 7315449139272470528}]
I20260806 06:38:45.022089 20360 mvcc.cc:205] Tried to move back new op lower bound from 7315449139272470528 to 7315449139198738432. Current Snapshot: MvccSnapshot[applied={T|T < 7315449139272470528}]
I20260806 06:38:45.022717 20359 mvcc.cc:205] Tried to move back new op lower bound from 7315449139272470528 to 7315449139198738432. Current Snapshot: MvccSnapshot[applied={T|T < 7315449139272470528}]
I20260806 06:38:45.024600 19686 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:38:45.024688 19686 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:38:45.025501 19686 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Sending DeleteTablet for 3 replicas of tablet b210d95771c748d4b6899787b15e066e
I20260806 06:38:45.025851 19858 tablet_service.cc:1514] Processing DeleteTablet for tablet b210d95771c748d4b6899787b15e066e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:45 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:55878
I20260806 06:38:45.025859 20229 tablet_service.cc:1514] Processing DeleteTablet for tablet b210d95771c748d4b6899787b15e066e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:45 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:49882
I20260806 06:38:45.025900 20044 tablet_service.cc:1514] Processing DeleteTablet for tablet b210d95771c748d4b6899787b15e066e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:45 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:46636
I20260806 06:38:45.026185 20375 tablet_replica.cc:330] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618: stopping tablet replica
I20260806 06:38:45.026201 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 19772
I20260806 06:38:45.026300 20375 raft_consensus.cc:2229] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:38:45.031527 20375 raft_consensus.cc:2258] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:38:45.032136 20374 tablet_replica.cc:330] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967: stopping tablet replica
I20260806 06:38:45.032303 20374 raft_consensus.cc:2229] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:38:45.032464 20374 raft_consensus.cc:2258] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:38:45.035300 20375 ts_tablet_manager.cc:1905] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:38:45.035385 20374 ts_tablet_manager.cc:1905] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:38:45.037292 20374 ts_tablet_manager.cc:1918] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:38:45.037292 20375 ts_tablet_manager.cc:1918] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:38:45.037386 20374 log.cc:1196] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/b210d95771c748d4b6899787b15e066e
I20260806 06:38:45.037386 20375 log.cc:1196] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/b210d95771c748d4b6899787b15e066e
I20260806 06:38:45.037717 20374 ts_tablet_manager.cc:1939] T b210d95771c748d4b6899787b15e066e P 63b22c163965490b85a29f556da28967: Deleting consensus metadata
I20260806 06:38:45.037717 20375 ts_tablet_manager.cc:1939] T b210d95771c748d4b6899787b15e066e P b9ec3df106514e309c5d903cc1536618: Deleting consensus metadata
I20260806 06:38:45.038239 19671 catalog_manager.cc:4841] TS 63b22c163965490b85a29f556da28967 (127.29.142.131:45201): tablet b210d95771c748d4b6899787b15e066e (table test-table [id=fe6a2181923e4c5a92d42e82acc64afc]) successfully deleted
I20260806 06:38:45.038215 19669 catalog_manager.cc:4841] TS b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525): tablet b210d95771c748d4b6899787b15e066e (table test-table [id=fe6a2181923e4c5a92d42e82acc64afc]) successfully deleted
W20260806 06:38:45.040052 19669 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:42987 (error 108)
I20260806 06:38:45.040246 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 19956
I20260806 06:38:45.040640 20334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:38:45.040519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42987 (local address 127.0.0.1:55904)
0806 06:38:45.040572 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:38:45.040601 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:42987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:38:45.040742 19669 catalog_manager.cc:4584] TS ee6b01b560b144b0a5d1f576796184db (127.29.142.129:42987): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b210d95771c748d4b6899787b15e066e: Network error: Client connection negotiation failed: client connection to 127.29.142.129:42987: connect: Connection refused (error 111)
I20260806 06:38:45.049263 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 20141
I20260806 06:38:45.061980 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 19646
I20260806 06:39:05.075448 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:43607
--webserver_interface=127.29.142.190
--webserver_port=34741
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:43607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:05.165012 20378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:05.165253 20378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:05.165277 20378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:05.166392 20378 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:39:05.166432 20378 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:39:05.166446 20378 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:39:05.166460 20378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:05.166472 20378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:05.166486 20378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:05.166498 20378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:05.166510 20378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:05.168043 20378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:43607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:43607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=34741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:05.168258 20378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:05.168511 20378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:05.178638 20394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:05.178709 20378 server_base.cc:1034] running on GCE node
W20260806 06:39:05.178620 20388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:05.178617 20387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:05.179209 20378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:05.179605 20378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:05.180752 20378 hybrid_clock.cc:648] HybridClock initialized: now 1785998345180733 us; error 44 us; skew 500 ppm
Aug 06 06:39:05 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998345, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:05.188648 20378 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:39:05.189210 20378 webserver.cc:460] Webserver started at http://127.29.142.190:34741/ using document root <none> and password file <none>
I20260806 06:39:05.189491 20378 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:05.189581 20378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:05.191151 20378 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:39:05.191910 20406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:05.192149 20378 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:05.192271 20378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "68c0d147b2804ee6a824cdebbe1f7414"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "4e280cf91bbc9ad7582664094f44c3c7"
server_key_iv: "0558091d65fcba87d086dbb6bc5207ff"
server_key_version: "encryptionkey@0"
I20260806 06:39:05.192731 20378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:05.208343 20378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:05.210638 20378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:05.210888 20378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:05.215775 20378 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:43607
I20260806 06:39:05.215837 20470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:43607 every 8 connection(s)
I20260806 06:39:05.216320 20378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:05.218169 20472 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:39:05.219074 20472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Bootstrap starting.
I20260806 06:39:05.222321 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 20378
I20260806 06:39:05.222728 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:42987
--local_ip_for_outbound_sockets=127.29.142.129
--tserver_master_addrs=127.29.142.190:43607
--webserver_port=36393
--webserver_interface=127.29.142.129
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:39:05.228950 20472 log.cc:824] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:05.231489 20472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:39:05.231896 20472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Bootstrap complete.
I20260806 06:39:05.234543 20472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } }
I20260806 06:39:05.234930 20472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c0d147b2804ee6a824cdebbe1f7414, State: Initialized, Role: FOLLOWER
I20260806 06:39:05.235142 20472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } }
I20260806 06:39:05.235209 20472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:05.235289 20472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:05.235378 20472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:39:05.236601 20472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } }
I20260806 06:39:05.236738 20472 leader_election.cc:302] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68c0d147b2804ee6a824cdebbe1f7414; no voters:
I20260806 06:39:05.236976 20472 leader_election.cc:288] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:39:05.237021 20477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:39:05.237170 20477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 2 LEADER]: Becoming Leader. State: Replica: 68c0d147b2804ee6a824cdebbe1f7414, State: Running, Role: LEADER
I20260806 06:39:05.237282 20477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } }
I20260806 06:39:05.237313 20472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:05.237613 20479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68c0d147b2804ee6a824cdebbe1f7414. Latest consensus state: current_term: 2 leader_uuid: "68c0d147b2804ee6a824cdebbe1f7414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } } }
I20260806 06:39:05.237855 20479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:05.237784 20478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "68c0d147b2804ee6a824cdebbe1f7414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } } }
I20260806 06:39:05.238119 20478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:05.238147 20485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:05.239264 20485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:39:05.239444 20485 catalog_manager.cc:1259] Loaded cluster ID: fdfb8ca979f3411289b2f1eb9d6b39cd
I20260806 06:39:05.239509 20485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:39:05.240631 20485 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:05.240863 20485 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Loaded TSK: 0
I20260806 06:39:05.257062 20485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Generated new TSK 1
I20260806 06:39:05.257236 20485 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:39:05.324569 20475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:05.324854 20475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:05.324903 20475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:05.326114 20475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:05.326197 20475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:05.326242 20475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:39:05.327885 20475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:42987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=36393
--tserver_master_addrs=127.29.142.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:05.328197 20475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:05.328505 20475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:05.334095 20511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:05.334297 20475 server_base.cc:1034] running on GCE node
W20260806 06:39:05.334087 20507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:05.334153 20506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:05.334755 20475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:05.335064 20475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:05.336278 20475 hybrid_clock.cc:648] HybridClock initialized: now 1785998345336261 us; error 37 us; skew 500 ppm
Aug 06 06:39:05 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998345, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:05.339591 20475 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:39:05.340152 20475 webserver.cc:460] Webserver started at http://127.29.142.129:36393/ using document root <none> and password file <none>
I20260806 06:39:05.340418 20475 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:05.340512 20475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:05.342751 20475 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:39:05.343582 20521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:05.343773 20475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:05.343880 20475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "ee6b01b560b144b0a5d1f576796184db"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "01f5139f8501a7a578173c4433c104a5"
server_key_iv: "1740bf3150849641c0c84924db62d7cd"
server_key_version: "encryptionkey@0"
I20260806 06:39:05.344292 20475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:05.357352 20475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:05.360325 20475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:05.360520 20475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:05.360858 20475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:05.361421 20535 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:39:05.363207 20475 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:39:05.363276 20475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:39:05.363323 20475 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:39:05.364245 20475 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:39:05.364298 20475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:39:05.364418 20535 tablet_bootstrap.cc:492] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Bootstrap starting.
I20260806 06:39:05.372051 20475 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:42987
I20260806 06:39:05.372555 20475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:39:05.373324 20682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:42987 every 8 connection(s)
I20260806 06:39:05.375864 20535 log.cc:824] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:05.379282 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 20475
I20260806 06:39:05.379623 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:34525
--local_ip_for_outbound_sockets=127.29.142.130
--tserver_master_addrs=127.29.142.190:43607
--webserver_port=34537
--webserver_interface=127.29.142.130
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:39:05.380096 20535 tablet_bootstrap.cc:492] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:39:05.380429 20535 tablet_bootstrap.cc:492] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Bootstrap complete.
I20260806 06:39:05.382128 20535 ts_tablet_manager.cc:1397] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.003s
Aug 06 06:39:05 dist-test-slave-3kk6 krb5kdc[19631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998345, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:05.383664 20535 raft_consensus.cc:348] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } } peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } }
I20260806 06:39:05.384023 20535 raft_consensus.cc:729] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee6b01b560b144b0a5d1f576796184db, State: Initialized, Role: FOLLOWER
I20260806 06:39:05.384169 20535 consensus_queue.cc:260] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } } peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } }
I20260806 06:39:05.384502 20535 ts_tablet_manager.cc:1428] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:39:05.388279 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.374047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:52939)
0806 06:39:05.374450 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:05.374498 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:05.375147 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:05.375757 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:05.375761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:05.376165 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:05.376369 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.376376 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.377636 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.377637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.378991 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.378994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.380934 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.380935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.381123 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.381881 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:05.381902 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:05.384062 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:05.386404 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:05.386412 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:05.386414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:05.386749 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:05.387063 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:05.387064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:05.387065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:05.387188 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:05.387647 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:05.387653 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:05.387794 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 06:39:05.387887 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 06:39:05.388058 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":136,"threads_started":1}
I20260806 06:39:05.388898 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.374135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:52939 (local address 127.29.142.190:43607)
0806 06:39:05.374353 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:05.374363 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:05.375329 (+ 966us) server_negotiation.cc:407] Connection header received
0806 06:39:05.375474 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:05.375480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:05.375534 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:05.375660 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:05.376521 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.377471 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.379973 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.380611 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.380693 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.384275 (+ 3582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:05.384280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:05.384280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:05.384294 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:05.386293 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:05.386840 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:05.386844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:05.386844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:05.386980 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:05.387253 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:05.387254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:05.387254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:05.387547 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:05.387619 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:05.388573 (+ 954us) server_negotiation.cc:299] Negotiation successful
0806 06:39:05.388716 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":85,"threads_started":1}
I20260806 06:39:05.389647 20684 heartbeater.cc:347] Connected to a master server at 127.29.142.190:43607
I20260806 06:39:05.389752 20684 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:05.389964 20684 heartbeater.cc:511] Master 127.29.142.190:43607 requested a full tablet report, sending...
I20260806 06:39:05.390916 20429 ts_manager.cc:194] Registered new tserver with Master: ee6b01b560b144b0a5d1f576796184db (127.29.142.129:42987)
I20260806 06:39:05.392168 20429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:52939
I20260806 06:39:05.393251 20684 heartbeater.cc:503] Master 127.29.142.190:43607 was elected leader, sending a full tablet report...
I20260806 06:39:05.397414 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.391961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.29.142.129:42987)
0806 06:39:05.394062 (+ 2101us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:05.394066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:05.394074 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:39:05.394103 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:05.394105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:05.394116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:05.394200 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:05.394393 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.395199 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.396400 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.396978 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.397081 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.397140 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:05.397297 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 06:39:05.397338 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2074,"thread_start_us":85,"threads_started":1}
I20260806 06:39:05.397413 20693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.391691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42987 (local address 127.0.0.1:44120)
0806 06:39:05.393827 (+ 2136us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:05.393865 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:05.393935 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:05.394221 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:05.394224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:05.394226 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:05.394293 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.394295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.395285 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.395286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.396290 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.396293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.397081 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.397082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.397169 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.397183 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:39:05.397279 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:39:05.397345 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2097,"thread_start_us":110,"threads_started":1}
I20260806 06:39:05.398141 20587 tablet_service.cc:1514] Processing DeleteTablet for tablet b210d95771c748d4b6899787b15e066e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:38:45 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:44120
I20260806 06:39:05.398743 20696 tablet_replica.cc:330] stopping tablet replica
I20260806 06:39:05.398877 20696 raft_consensus.cc:2229] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:39:05.398968 20696 raft_consensus.cc:2258] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:05.399307 20696 ts_tablet_manager.cc:1905] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:05.401401 20696 ts_tablet_manager.cc:1918] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:05.401468 20696 log.cc:1196] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal/wals/b210d95771c748d4b6899787b15e066e
I20260806 06:39:05.401708 20696 ts_tablet_manager.cc:1939] T b210d95771c748d4b6899787b15e066e P ee6b01b560b144b0a5d1f576796184db: Deleting consensus metadata
I20260806 06:39:05.402330 20408 catalog_manager.cc:4841] TS ee6b01b560b144b0a5d1f576796184db (127.29.142.129:42987): tablet b210d95771c748d4b6899787b15e066e (table test-table [id=fe6a2181923e4c5a92d42e82acc64afc]) successfully deleted
W20260806 06:39:05.473101 20691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:05.473380 20691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:05.473424 20691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:05.474695 20691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:05.474776 20691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:05.474821 20691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:39:05.476393 20691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:34525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=34537
--tserver_master_addrs=127.29.142.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:05.476696 20691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:05.476963 20691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:39:05.482615 20691 server_base.cc:1034] running on GCE node
W20260806 06:39:05.482584 20705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:05.482635 20701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:05.482584 20703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:05.483117 20691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:05.483397 20691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:05.484573 20691 hybrid_clock.cc:648] HybridClock initialized: now 1785998345484560 us; error 50 us; skew 500 ppm
Aug 06 06:39:05 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998345, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:05.487442 20691 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:39:05.487954 20691 webserver.cc:460] Webserver started at http://127.29.142.130:34537/ using document root <none> and password file <none>
I20260806 06:39:05.488243 20691 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:05.488340 20691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:05.489827 20691 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:39:05.490479 20713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:05.490710 20691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:39:05.490821 20691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "b9ec3df106514e309c5d903cc1536618"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "6f951ecb8420a8294127d158e48256a2"
server_key_iv: "66630582ca242df199370a8ffd45ce63"
server_key_version: "encryptionkey@0"
I20260806 06:39:05.491216 20691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:05.512939 20691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:05.515693 20691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:05.515877 20691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:05.516135 20691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:05.516477 20691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:05.516532 20691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:05.516569 20691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:05.516618 20691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:05.522914 20691 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:34525
I20260806 06:39:05.522956 20869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:34525 every 8 connection(s)
I20260806 06:39:05.523389 20691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:39:05.529075 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 20691
I20260806 06:39:05.529423 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:45201
--local_ip_for_outbound_sockets=127.29.142.131
--tserver_master_addrs=127.29.142.190:43607
--webserver_port=45131
--webserver_interface=127.29.142.131
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:39:05 dist-test-slave-3kk6 krb5kdc[19631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998345, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:05.535487 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.524394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:49943 (local address 127.29.142.190:43607)
0806 06:39:05.524470 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:05.524472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:05.524637 (+ 165us) server_negotiation.cc:407] Connection header received
0806 06:39:05.525135 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:05.525136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:05.525149 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:05.525218 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:05.526257 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.526883 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.527999 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.528215 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.528291 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.531212 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:05.531214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:05.531214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:05.531225 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:05.533192 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:05.533906 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:05.533907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:05.533908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:05.533982 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:05.534476 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:05.534477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:05.534477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:05.534690 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:05.534777 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:05.535201 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 06:39:05.535333 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:39:05.535720 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.524310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:49943)
0806 06:39:05.524483 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:05.524530 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:05.525043 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:05.525276 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:05.525280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:05.525885 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:05.526104 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.526110 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.527024 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.527025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.527883 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.527885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.528312 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.528313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.528423 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.529091 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:05.529100 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:05.531078 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:05.533342 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:05.533353 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:05.533356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:05.533780 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:05.534198 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:05.534200 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:05.534201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:05.534369 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:05.534784 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:05.534791 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:05.535014 (+ 223us) client_negotiation.cc:769] Sending connection context
0806 06:39:05.535170 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 06:39:05.535404 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260806 06:39:05.536499 20870 heartbeater.cc:347] Connected to a master server at 127.29.142.190:43607
I20260806 06:39:05.536633 20870 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:05.536844 20870 heartbeater.cc:511] Master 127.29.142.190:43607 requested a full tablet report, sending...
I20260806 06:39:05.537371 20429 ts_manager.cc:194] Registered new tserver with Master: b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525)
I20260806 06:39:05.537904 20429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:49943
W20260806 06:39:05.622805 20873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:05.623111 20873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:05.623164 20873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:05.624519 20873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:05.624617 20873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:05.624673 20873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:39:05.626533 20873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:45201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=45131
--tserver_master_addrs=127.29.142.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:05.626900 20873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:05.627205 20873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:05.632974 20881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:05.632982 20880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:05.632983 20873 server_base.cc:1034] running on GCE node
W20260806 06:39:05.633002 20885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:05.633515 20873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:05.633783 20873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:05.634933 20873 hybrid_clock.cc:648] HybridClock initialized: now 1785998345634922 us; error 32 us; skew 500 ppm
Aug 06 06:39:05 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998345, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:05.638056 20873 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:39:05.638581 20873 webserver.cc:460] Webserver started at http://127.29.142.131:45131/ using document root <none> and password file <none>
I20260806 06:39:05.638866 20873 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:05.638955 20873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:05.640376 20873 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:05.641011 20893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:05.641312 20873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:05.641399 20873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "63b22c163965490b85a29f556da28967"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "5e3071932f76c03300e8cf5b16127aff"
server_key_iv: "46ace4dee7c54a8c47c8d4a03b8ff094"
server_key_version: "encryptionkey@0"
I20260806 06:39:05.641819 20873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:05.654901 20873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:05.657189 20873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:05.657366 20873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:05.657626 20873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:05.657969 20873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:05.658026 20873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:05.658071 20873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:05.658099 20873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:05.664340 20873 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:45201
I20260806 06:39:05.664389 21045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:45201 every 8 connection(s)
I20260806 06:39:05.664775 20873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:39:05.666528 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 20873
Aug 06 06:39:05 dist-test-slave-3kk6 krb5kdc[19631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998345, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:05.674355 21052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.667059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47774 (local address 127.29.142.190:43607)
0806 06:39:05.667181 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:05.667183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:05.667314 (+ 131us) server_negotiation.cc:407] Connection header received
0806 06:39:05.667389 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:05.667390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:05.667398 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:05.667459 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:05.668283 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.668763 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.669525 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.669733 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.669811 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.671272 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:05.671273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:05.671273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:05.671280 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:05.673105 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:05.673471 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:05.673473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:05.673473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:05.673538 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:05.673753 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:05.673754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:05.673754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:05.673878 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:05.674098 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:05.674137 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 06:39:05.674235 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":78,"threads_started":1}
I20260806 06:39:05.675436 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.665575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:45095 (local address 127.29.142.190:43607)
0806 06:39:05.665650 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:05.665652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:05.665903 (+ 251us) server_negotiation.cc:407] Connection header received
0806 06:39:05.666427 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:05.666429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:05.666443 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:05.666524 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:05.668043 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.668567 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.669612 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.669808 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.669885 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.672406 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:05.672408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:05.672408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:05.672416 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:05.673912 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:05.674385 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:05.674386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:05.674387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:05.674443 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:05.674754 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:05.674755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:05.674755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:05.674869 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:05.674953 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:05.675311 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 06:39:05.675378 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:39:05.675724 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.665576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:45095)
0806 06:39:05.665770 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:05.665809 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:05.666324 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:05.666620 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:05.666624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:05.667618 (+ 994us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:05.667947 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.667955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.668666 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.668667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.669501 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.669504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.669896 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.669897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.670031 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.670624 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:05.670633 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:05.672302 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:05.674043 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:05.674050 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:05.674053 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:05.674310 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:05.674512 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:05.674514 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:05.674514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:05.674691 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:05.674956 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:05.674981 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:05.675153 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 06:39:05.675265 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 06:39:05.675429 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260806 06:39:05.676162 21047 heartbeater.cc:347] Connected to a master server at 127.29.142.190:43607
I20260806 06:39:05.676275 21047 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:05.676467 21047 heartbeater.cc:511] Master 127.29.142.190:43607 requested a full tablet report, sending...
I20260806 06:39:05.676873 20429 ts_manager.cc:194] Registered new tserver with Master: 63b22c163965490b85a29f556da28967 (127.29.142.131:45201)
I20260806 06:39:05.677218 20429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:45095
I20260806 06:39:05.686213 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:39:05.689065 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.686694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47780 (local address 127.29.142.190:43607)
0806 06:39:05.686753 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:05.686755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:05.686840 (+ 85us) server_negotiation.cc:407] Connection header received
0806 06:39:05.686896 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:05.686897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:05.686936 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:05.687011 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:39:05.687142 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.687622 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.688384 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.688638 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.688695 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.688841 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:39:05.688888 (+ 47us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 06:39:05.689012 (+ 124us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47780: token expired
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 06:39:05.689117 20687 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47780: token expired
W20260806 06:39:05.689316 20322 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.142.190:43607)
I20260806 06:39:05.695613 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.689621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47790 (local address 127.29.142.190:43607)
0806 06:39:05.689658 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:05.689660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:05.689667 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:39:05.689693 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:05.689694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:05.689701 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:05.689747 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:05.690213 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.690638 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.691386 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.691556 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.691605 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.692867 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:05.692868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:05.692869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:05.692875 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:05.694412 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:05.694900 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:05.694901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:05.694901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:05.694958 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:05.695171 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:05.695172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:05.695172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:05.695311 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:05.695389 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:05.695484 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 06:39:05.695548 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:39:05.696954 20429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:39:05.698534 20429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:39:05.701035 20587 tablet_service.cc:1467] Processing CreateTablet for tablet 1890ebd36edc4f4abfb76140cce0b02b (DEFAULT_TABLE table=test-table [id=1e54115992554d31a6d2e8aa2883a74a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:05.701294 20587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1890ebd36edc4f4abfb76140cce0b02b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:05.703024 20535 tablet_bootstrap.cc:492] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: Bootstrap starting.
I20260806 06:39:05.703384 20535 tablet_bootstrap.cc:654] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:05.704209 20535 tablet_bootstrap.cc:492] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: No bootstrap required, opened a new log
I20260806 06:39:05.704263 20535 ts_tablet_manager.cc:1397] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:05.704375 20535 raft_consensus.cc:348] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:05.704445 20535 raft_consensus.cc:374] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:05.704494 20535 raft_consensus.cc:729] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee6b01b560b144b0a5d1f576796184db, State: Initialized, Role: FOLLOWER
I20260806 06:39:05.704535 20535 consensus_queue.cc:260] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:05.704643 20535 ts_tablet_manager.cc:1428] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:05.705029 20693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.700660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:34525 (local address 127.0.0.1:60508)
0806 06:39:05.700716 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:05.700730 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:05.700806 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:05.701193 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:05.701194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:05.701196 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:05.701247 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.701248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.702592 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.702593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.703682 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.703686 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.704822 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.704823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.704894 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.704899 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:39:05.704935 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 06:39:05.704976 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 06:39:05.705464 21056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.700778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60508 (local address 127.29.142.130:34525)
0806 06:39:05.700999 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:05.701004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:05.701016 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:39:05.701059 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:05.701061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:05.701079 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:05.701324 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:05.701429 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.702481 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.703867 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.704712 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.705231 (+ 519us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.705318 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:05.705350 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 06:39:05.705395 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":111,"threads_started":1}
I20260806 06:39:05.706135 20780 tablet_service.cc:1467] Processing CreateTablet for tablet 1890ebd36edc4f4abfb76140cce0b02b (DEFAULT_TABLE table=test-table [id=1e54115992554d31a6d2e8aa2883a74a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:05.706539 20780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1890ebd36edc4f4abfb76140cce0b02b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:05.706761 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.700756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:45201 (local address 127.0.0.1:35828)
0806 06:39:05.703081 (+ 2325us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:05.703095 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:05.703152 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:05.703389 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:05.703389 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:05.703391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:05.703442 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.703443 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.704656 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.704657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.705744 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.705747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.706454 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.706456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.706523 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.706528 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:39:05.706603 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:39:05.706678 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2309,"thread_start_us":98,"threads_started":1}
I20260806 06:39:05.706827 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.701021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35828 (local address 127.29.142.131:45201)
0806 06:39:05.701504 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:05.701508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:05.703218 (+ 1710us) server_negotiation.cc:407] Connection header received
0806 06:39:05.703284 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:05.703287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:05.703301 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:05.703392 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:05.703491 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.704550 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.705839 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.706363 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.706445 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.706485 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:05.706635 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 06:39:05.706706 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":111,"threads_started":1}
I20260806 06:39:05.709619 21060 tablet_bootstrap.cc:492] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618: Bootstrap starting.
I20260806 06:39:05.710243 20962 tablet_service.cc:1467] Processing CreateTablet for tablet 1890ebd36edc4f4abfb76140cce0b02b (DEFAULT_TABLE table=test-table [id=1e54115992554d31a6d2e8aa2883a74a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:05.710495 21060 tablet_bootstrap.cc:654] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:05.710610 20962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1890ebd36edc4f4abfb76140cce0b02b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:05.711107 21060 log.cc:824] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:05.712041 21060 tablet_bootstrap.cc:492] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618: No bootstrap required, opened a new log
I20260806 06:39:05.712133 21060 ts_tablet_manager.cc:1397] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 06:39:05.713937 21062 tablet_bootstrap.cc:492] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967: Bootstrap starting.
I20260806 06:39:05.713891 21060 raft_consensus.cc:348] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:05.714111 21060 raft_consensus.cc:374] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:05.714159 21060 raft_consensus.cc:729] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ec3df106514e309c5d903cc1536618, State: Initialized, Role: FOLLOWER
I20260806 06:39:05.714293 21060 consensus_queue.cc:260] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:05.714695 21060 ts_tablet_manager.cc:1428] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:39:05.714701 21062 tablet_bootstrap.cc:654] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:05.714725 20870 heartbeater.cc:503] Master 127.29.142.190:43607 was elected leader, sending a full tablet report...
I20260806 06:39:05.715314 21062 log.cc:824] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:05.716236 21062 tablet_bootstrap.cc:492] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967: No bootstrap required, opened a new log
I20260806 06:39:05.716331 21062 ts_tablet_manager.cc:1397] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:39:05.717777 21062 raft_consensus.cc:348] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:05.718008 21062 raft_consensus.cc:374] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:05.718065 21062 raft_consensus.cc:729] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b22c163965490b85a29f556da28967, State: Initialized, Role: FOLLOWER
I20260806 06:39:05.718197 21062 consensus_queue.cc:260] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:05.718545 21062 ts_tablet_manager.cc:1428] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 06:39:05.718578 21047 heartbeater.cc:503] Master 127.29.142.190:43607 was elected leader, sending a full tablet report...
W20260806 06:39:05.774144 20871 tablet.cc:2369] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:39:05.796136 21066 raft_consensus.cc:482] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:39:05.796276 21066 raft_consensus.cc:504] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:05.796739 21066 leader_election.cc:288] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525), ee6b01b560b144b0a5d1f576796184db (127.29.142.129:42987)
I20260806 06:39:05.799760 21051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.796943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:34525 (local address 127.29.142.131:51933)
0806 06:39:05.797000 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:05.797012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:05.797090 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:05.797235 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:05.797235 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:05.797236 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:05.797307 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.797309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.798035 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.798037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.798886 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.798889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.799476 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.799477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.799566 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.799569 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:39:05.799635 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:39:05.799679 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:39:05.799760 21056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.797005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:51933 (local address 127.29.142.130:34525)
0806 06:39:05.797086 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:05.797088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:05.797100 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:39:05.797159 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:05.797160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:05.797170 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:05.797221 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:05.797390 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.797936 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.798983 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.799377 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.799465 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.799475 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:05.799640 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 06:39:05.799689 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 06:39:05.800083 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.797197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:41935 (local address 127.29.142.129:42987)
0806 06:39:05.797246 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:05.797249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:05.797398 (+ 149us) server_negotiation.cc:407] Connection header received
0806 06:39:05.797479 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:05.797480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:05.797490 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:05.797609 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:05.797833 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.798307 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.799288 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.799657 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.799746 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.799754 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:05.799962 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 06:39:05.800018 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:39:05.800143 20808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1890ebd36edc4f4abfb76140cce0b02b" candidate_uuid: "63b22c163965490b85a29f556da28967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ec3df106514e309c5d903cc1536618" is_pre_election: true
I20260806 06:39:05.800338 21067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.797119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42987 (local address 127.29.142.131:41935)
0806 06:39:05.797305 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:05.797315 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:05.797406 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:05.797602 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:05.797603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:05.797605 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:05.797717 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.797720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.798408 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.798409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.799185 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:05.799188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.799748 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.799749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:05.799857 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.799861 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:39:05.800181 (+ 320us) client_negotiation.cc:240] Negotiation successful
0806 06:39:05.800235 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":128,"threads_started":1}
I20260806 06:39:05.800374 20808 raft_consensus.cc:2454] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63b22c163965490b85a29f556da28967 in term 0.
I20260806 06:39:05.800633 20895 leader_election.cc:302] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b22c163965490b85a29f556da28967, b9ec3df106514e309c5d903cc1536618; no voters:
I20260806 06:39:05.800664 20618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1890ebd36edc4f4abfb76140cce0b02b" candidate_uuid: "63b22c163965490b85a29f556da28967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee6b01b560b144b0a5d1f576796184db" is_pre_election: true
I20260806 06:39:05.800808 20618 raft_consensus.cc:2454] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63b22c163965490b85a29f556da28967 in term 0.
I20260806 06:39:05.800863 21066 raft_consensus.cc:2781] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:39:05.800957 21066 raft_consensus.cc:482] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:39:05.801002 21066 raft_consensus.cc:3037] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:05.801792 21066 raft_consensus.cc:504] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:05.801918 21066 leader_election.cc:288] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [CANDIDATE]: Term 1 election: Requested vote from peers b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525), ee6b01b560b144b0a5d1f576796184db (127.29.142.129:42987)
I20260806 06:39:05.802124 20618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1890ebd36edc4f4abfb76140cce0b02b" candidate_uuid: "63b22c163965490b85a29f556da28967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee6b01b560b144b0a5d1f576796184db"
I20260806 06:39:05.802143 20808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1890ebd36edc4f4abfb76140cce0b02b" candidate_uuid: "63b22c163965490b85a29f556da28967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9ec3df106514e309c5d903cc1536618"
I20260806 06:39:05.802225 20618 raft_consensus.cc:3037] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:05.802233 20808 raft_consensus.cc:3037] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:05.803316 20808 raft_consensus.cc:2454] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63b22c163965490b85a29f556da28967 in term 1.
I20260806 06:39:05.803543 20895 leader_election.cc:302] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b22c163965490b85a29f556da28967, b9ec3df106514e309c5d903cc1536618; no voters:
I20260806 06:39:05.803670 21066 raft_consensus.cc:2781] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:05.803660 20618 raft_consensus.cc:2454] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63b22c163965490b85a29f556da28967 in term 1.
I20260806 06:39:05.803901 21066 raft_consensus.cc:686] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 1 LEADER]: Becoming Leader. State: Replica: 63b22c163965490b85a29f556da28967, State: Running, Role: LEADER
I20260806 06:39:05.804050 21066 consensus_queue.cc:237] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:05.805003 20429 catalog_manager.cc:5495] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 reported cstate change: term changed from 0 to 1, leader changed from <none> to 63b22c163965490b85a29f556da28967 (127.29.142.131). New cstate: current_term: 1 leader_uuid: "63b22c163965490b85a29f556da28967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } health_report { overall_health: UNKNOWN } } }
I20260806 06:39:05.824118 21057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:05.821112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35838 (local address 127.29.142.131:45201)
0806 06:39:05.821184 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:05.821186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:05.821194 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:39:05.821233 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:05.821233 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:05.821253 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:05.821330 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:39:05.821505 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.821987 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.823102 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:05.823296 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:05.823370 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:05.823520 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:39:05.823633 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:39:05.823968 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:05.824018 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 06:39:05.824046 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:39:05.826345 20808 raft_consensus.cc:1261] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618 [term 1 FOLLOWER]: Refusing update from remote peer 63b22c163965490b85a29f556da28967: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:05.826385 20618 raft_consensus.cc:1261] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Refusing update from remote peer 63b22c163965490b85a29f556da28967: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:05.826877 21068 consensus_queue.cc:1035] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:05.827080 21066 consensus_queue.cc:1035] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:05.832661 20429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:05.832773 20429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:05.833516 20429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Sending DeleteTablet for 3 replicas of tablet 1890ebd36edc4f4abfb76140cce0b02b
I20260806 06:39:05.833899 20587 tablet_service.cc:1514] Processing DeleteTablet for tablet 1890ebd36edc4f4abfb76140cce0b02b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:05 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:44120
I20260806 06:39:05.833912 20962 tablet_service.cc:1514] Processing DeleteTablet for tablet 1890ebd36edc4f4abfb76140cce0b02b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:05 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:35828
I20260806 06:39:05.834008 20696 tablet_replica.cc:330] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: stopping tablet replica
I20260806 06:39:05.833977 20780 tablet_service.cc:1514] Processing DeleteTablet for tablet 1890ebd36edc4f4abfb76140cce0b02b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:05 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:60508
I20260806 06:39:05.834053 20696 raft_consensus.cc:2229] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:39:05.834105 20696 raft_consensus.cc:2258] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:05.834242 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 20475
I20260806 06:39:05.834409 20696 ts_tablet_manager.cc:1905] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:05.847404 21089 tablet_replica.cc:330] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618: stopping tablet replica
I20260806 06:39:05.847632 21089 raft_consensus.cc:2229] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:39:05.847810 21089 raft_consensus.cc:2258] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:05.847819 21088 tablet_replica.cc:330] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967: stopping tablet replica
I20260806 06:39:05.848031 21088 raft_consensus.cc:2229] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:39:05.848186 21088 raft_consensus.cc:2258] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:05.848836 21088 ts_tablet_manager.cc:1905] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:05.850790 21088 ts_tablet_manager.cc:1918] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:05.850884 21088 log.cc:1196] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/1890ebd36edc4f4abfb76140cce0b02b
I20260806 06:39:05.851210 21088 ts_tablet_manager.cc:1939] T 1890ebd36edc4f4abfb76140cce0b02b P 63b22c163965490b85a29f556da28967: Deleting consensus metadata
I20260806 06:39:05.851758 21089 ts_tablet_manager.cc:1905] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:05.851758 20411 catalog_manager.cc:4841] TS 63b22c163965490b85a29f556da28967 (127.29.142.131:45201): tablet 1890ebd36edc4f4abfb76140cce0b02b (table test-table [id=1e54115992554d31a6d2e8aa2883a74a]) successfully deleted
W20260806 06:39:05.853266 20408 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:42987 (error 108)
W20260806 06:39:05.854024 21055 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:39:05.853705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42987 (local address 127.0.0.1:44134)
0806 06:39:05.853773 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:05.853790 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:05.853845 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:05.853975 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:42987: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 06:39:05.854038 21089 ts_tablet_manager.cc:1918] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:05.854128 21089 log.cc:1196] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/1890ebd36edc4f4abfb76140cce0b02b
I20260806 06:39:05.854475 21089 ts_tablet_manager.cc:1939] T 1890ebd36edc4f4abfb76140cce0b02b P b9ec3df106514e309c5d903cc1536618: Deleting consensus metadata
W20260806 06:39:05.854161 20408 catalog_manager.cc:4584] TS ee6b01b560b144b0a5d1f576796184db (127.29.142.129:42987): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1890ebd36edc4f4abfb76140cce0b02b: Network error: Client connection negotiation failed: client connection to 127.29.142.129:42987: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:39:05.855041 20408 catalog_manager.cc:4841] TS b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525): tablet 1890ebd36edc4f4abfb76140cce0b02b (table test-table [id=1e54115992554d31a6d2e8aa2883a74a]) successfully deleted
I20260806 06:39:05.855197 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 20691
I20260806 06:39:05.872411 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 20873
I20260806 06:39:05.888638 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 20378
I20260806 06:39:25.897343 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:43607
--webserver_interface=127.29.142.190
--webserver_port=34741
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:43607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:25.983847 21092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:25.984079 21092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:25.984104 21092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:25.985301 21092 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:39:25.985347 21092 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:39:25.985359 21092 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:39:25.985371 21092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:25.985384 21092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:25.985424 21092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:25.985457 21092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:25.985476 21092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:25.987035 21092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:43607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:43607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=34741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:25.987308 21092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:25.987578 21092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:25.993283 21102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:25.993359 21108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:25.993371 21092 server_base.cc:1034] running on GCE node
W20260806 06:39:25.993280 21099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:25.993706 21092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:25.993983 21092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:25.995122 21092 hybrid_clock.cc:648] HybridClock initialized: now 1785998365995120 us; error 42 us; skew 500 ppm
Aug 06 06:39:25 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998365, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:26.002732 21092 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:39:26.003294 21092 webserver.cc:460] Webserver started at http://127.29.142.190:34741/ using document root <none> and password file <none>
I20260806 06:39:26.003548 21092 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:26.003599 21092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:26.005075 21092 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:39:26.005844 21117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:26.006068 21092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:26.006179 21092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "68c0d147b2804ee6a824cdebbe1f7414"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "4e280cf91bbc9ad7582664094f44c3c7"
server_key_iv: "0558091d65fcba87d086dbb6bc5207ff"
server_key_version: "encryptionkey@0"
I20260806 06:39:26.006580 21092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:26.028827 21092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:26.031147 21092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:26.031350 21092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:26.035926 21092 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:43607
I20260806 06:39:26.035970 21196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:43607 every 8 connection(s)
I20260806 06:39:26.036377 21092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:26.037910 21197 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:39:26.038578 21197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Bootstrap starting.
I20260806 06:39:26.043943 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 21092
I20260806 06:39:26.044327 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:42987
--local_ip_for_outbound_sockets=127.29.142.129
--tserver_master_addrs=127.29.142.190:43607
--webserver_port=36393
--webserver_interface=127.29.142.129
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:39:26.047996 21197 log.cc:824] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:26.051859 21197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:39:26.052314 21197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Bootstrap complete.
I20260806 06:39:26.056535 21197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } }
I20260806 06:39:26.056963 21197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c0d147b2804ee6a824cdebbe1f7414, State: Initialized, Role: FOLLOWER
I20260806 06:39:26.057169 21197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } }
I20260806 06:39:26.057271 21197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:26.057334 21197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:26.057407 21197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:39:26.058876 21197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } }
I20260806 06:39:26.059007 21197 leader_election.cc:302] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68c0d147b2804ee6a824cdebbe1f7414; no voters:
I20260806 06:39:26.059211 21197 leader_election.cc:288] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 06:39:26.059302 21201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 06:39:26.059504 21201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [term 3 LEADER]: Becoming Leader. State: Replica: 68c0d147b2804ee6a824cdebbe1f7414, State: Running, Role: LEADER
I20260806 06:39:26.059525 21197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:26.059613 21201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } }
I20260806 06:39:26.059954 21202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "68c0d147b2804ee6a824cdebbe1f7414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } } }
I20260806 06:39:26.060045 21202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:26.060353 21203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68c0d147b2804ee6a824cdebbe1f7414. Latest consensus state: current_term: 3 leader_uuid: "68c0d147b2804ee6a824cdebbe1f7414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c0d147b2804ee6a824cdebbe1f7414" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 43607 } } }
I20260806 06:39:26.060508 21203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:26.060634 21211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:26.061530 21211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:39:26.061648 21211 catalog_manager.cc:1259] Loaded cluster ID: fdfb8ca979f3411289b2f1eb9d6b39cd
I20260806 06:39:26.061707 21211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:39:26.062767 21211 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:26.062978 21211 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Loaded TSK: 0
I20260806 06:39:26.063050 21211 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 68c0d147b2804ee6a824cdebbe1f7414: Loaded TSK: 1
F20260806 06:39:26.063306 21211 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f669fa8e9ad google::LogMessage::Fail() at ??:0
@ 0x7f669fa9188a google::LogMessage::SendToLog() at ??:0
@ 0x7f669fa8e4de google::LogMessage::Flush() at ??:0
@ 0x7f669fa8fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f66a21fb104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f66a21f8d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f66a79e81cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f66a79e848e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f66a79e8594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f66a79aec48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f66a79b8ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f66a195c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f66a1951954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f66a27b56db start_thread at ??:0
@ 0x7f669e58c61f clone at ??:0
W20260806 06:39:26.161088 21199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:26.161321 21199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:26.161347 21199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:26.162529 21199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:26.162573 21199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:26.162611 21199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:39:26.164124 21199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:42987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=36393
--tserver_master_addrs=127.29.142.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:26.164373 21199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:26.164649 21199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:26.169970 21248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:26.170011 21254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:26.170013 21252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:26.170336 21199 server_base.cc:1034] running on GCE node
I20260806 06:39:26.170545 21199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:26.170817 21199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:26.171985 21199 hybrid_clock.cc:648] HybridClock initialized: now 1785998366171973 us; error 34 us; skew 500 ppm
Aug 06 06:39:26 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:26.174629 21199 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:39:26.175155 21199 webserver.cc:460] Webserver started at http://127.29.142.129:36393/ using document root <none> and password file <none>
I20260806 06:39:26.175417 21199 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:26.175493 21199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:26.176978 21199 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:26.177860 21262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:26.178045 21199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:26.178141 21199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "ee6b01b560b144b0a5d1f576796184db"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "01f5139f8501a7a578173c4433c104a5"
server_key_iv: "1740bf3150849641c0c84924db62d7cd"
server_key_version: "encryptionkey@0"
I20260806 06:39:26.178500 21199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:26.189417 21199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:26.192026 21199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:26.192189 21199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:26.192466 21199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:26.192965 21272 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:39:26.195087 21199 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:39:26.195158 21199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:39:26.195207 21199 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:39:26.195895 21199 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:39:26.195958 21199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:39:26.196036 21272 tablet_bootstrap.cc:492] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: Bootstrap starting.
I20260806 06:39:26.198014 21272 log.cc:824] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:26.202225 21272 tablet_bootstrap.cc:492] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:39:26.202831 21272 tablet_bootstrap.cc:492] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: Bootstrap complete.
I20260806 06:39:26.203091 21272 ts_tablet_manager.cc:1397] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260806 06:39:26.204571 21199 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:42987
I20260806 06:39:26.205055 21199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:39:26.206333 21272 raft_consensus.cc:348] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:26.206705 21272 raft_consensus.cc:729] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee6b01b560b144b0a5d1f576796184db, State: Initialized, Role: FOLLOWER
I20260806 06:39:26.206852 21272 consensus_queue.cc:260] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:26.207168 21272 ts_tablet_manager.cc:1428] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260806 06:39:26.207355 21419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:42987 every 8 connection(s)
I20260806 06:39:26.212675 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 21199
I20260806 06:39:26.213029 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:34525
--local_ip_for_outbound_sockets=127.29.142.130
--tserver_master_addrs=127.29.142.190:43607
--webserver_port=34537
--webserver_interface=127.29.142.130
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:26.275309 21425 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:39:26.207098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:41057)
0806 06:39:26.207330 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:26.207383 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:26.207556 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:26.275061 (+ 67505us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":82,"threads_started":1}
W20260806 06:39:26.275612 21420 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:43607 (0 consecutive failures): Network error: Failed to ping master at 127.29.142.190:43607: Client connection negotiation failed: client connection to 127.29.142.190:43607: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:39:26.275980 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:26.275849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:55113)
0806 06:39:26.275911 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:26.275952 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 06:39:26.276360 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:26.276265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:53127)
0806 06:39:26.276319 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:26.276345 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 06:39:26.276489 21420 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:39:26.301445 21426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:26.301700 21426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:26.301743 21426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:26.303007 21426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:26.303072 21426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:26.303115 21426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:39:26.304657 21426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:34525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=34537
--tserver_master_addrs=127.29.142.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:26.304941 21426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:26.305184 21426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:26.310470 21436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:26.310510 21432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:26.310597 21431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:26.310917 21426 server_base.cc:1034] running on GCE node
I20260806 06:39:26.311148 21426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:26.311414 21426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:26.312582 21426 hybrid_clock.cc:648] HybridClock initialized: now 1785998366312556 us; error 43 us; skew 500 ppm
Aug 06 06:39:26 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:26.315444 21426 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:39:26.315951 21426 webserver.cc:460] Webserver started at http://127.29.142.130:34537/ using document root <none> and password file <none>
I20260806 06:39:26.316211 21426 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:26.316287 21426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:26.317689 21426 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:39:26.318418 21447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:26.318593 21426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:26.318756 21426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "b9ec3df106514e309c5d903cc1536618"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "6f951ecb8420a8294127d158e48256a2"
server_key_iv: "66630582ca242df199370a8ffd45ce63"
server_key_version: "encryptionkey@0"
I20260806 06:39:26.319166 21426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:26.333385 21426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:26.335938 21426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:26.336119 21426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:26.336380 21426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:26.336709 21426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:26.336779 21426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:26.336851 21426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:26.336884 21426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:26.343132 21426 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:34525
I20260806 06:39:26.343178 21609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:34525 every 8 connection(s)
I20260806 06:39:26.343551 21426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:39:26.344624 21613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:26.344238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:41407)
0806 06:39:26.344405 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:26.344480 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
W20260806 06:39:26.345049 21611 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:43607 (0 consecutive failures): Network error: Failed to ping master at 127.29.142.190:43607: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
I20260806 06:39:26.345381 21613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:26.345284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:46879)
0806 06:39:26.345333 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:26.345361 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:39:26.345680 21613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:26.345604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:35677)
0806 06:39:26.345645 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:26.345667 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 06:39:26.345824 21611 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:39:26.352248 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 21426
I20260806 06:39:26.352577 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:45201
--local_ip_for_outbound_sockets=127.29.142.131
--tserver_master_addrs=127.29.142.190:43607
--webserver_port=45131
--webserver_interface=127.29.142.131
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:26.436939 21614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:26.437167 21614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:26.437192 21614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:26.438369 21614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:26.438416 21614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:26.438443 21614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:39:26.440002 21614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:45201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=45131
--tserver_master_addrs=127.29.142.190:43607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:26.440268 21614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:26.440510 21614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:26.445791 21619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:26.445797 21622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:26.445854 21620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:26.446063 21614 server_base.cc:1034] running on GCE node
I20260806 06:39:26.446297 21614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:26.446573 21614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:26.447784 21614 hybrid_clock.cc:648] HybridClock initialized: now 1785998366447782 us; error 51 us; skew 500 ppm
Aug 06 06:39:26 dist-test-slave-3kk6 krb5kdc[19631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:26.450410 21614 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:39:26.450950 21614 webserver.cc:460] Webserver started at http://127.29.142.131:45131/ using document root <none> and password file <none>
I20260806 06:39:26.451170 21614 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:26.451215 21614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:26.452608 21614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:26.453316 21633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:26.453521 21614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:26.453619 21614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "63b22c163965490b85a29f556da28967"
format_stamp: "Formatted at 2026-08-06 06:38:44 on dist-test-slave-3kk6"
server_key: "5e3071932f76c03300e8cf5b16127aff"
server_key_iv: "46ace4dee7c54a8c47c8d4a03b8ff094"
server_key_version: "encryptionkey@0"
I20260806 06:39:26.453984 21614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:26.487947 21614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:26.490319 21614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:26.490517 21614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:26.490885 21614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:26.491312 21614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:26.491369 21614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:26.491415 21614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:26.491444 21614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:26.497510 21614 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:45201
I20260806 06:39:26.497695 21784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:45201 every 8 connection(s)
I20260806 06:39:26.497991 21614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:39:26.498796 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 21614
I20260806 06:39:26.499287 21788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:26.498793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:49551)
0806 06:39:26.499024 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:26.499128 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":99,"threads_started":1}
W20260806 06:39:26.499725 21785 heartbeater.cc:644] Failed to heartbeat to 127.29.142.190:43607 (0 consecutive failures): Network error: Failed to ping master at 127.29.142.190:43607: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
I20260806 06:39:26.500033 21788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:26.499936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:41031)
0806 06:39:26.499987 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:26.500015 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:39:26.500326 21788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:26.500253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:45359)
0806 06:39:26.500290 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:26.500312 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 06:39:26.500484 21785 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:39:27.277252 21791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:27.276954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:43907)
0806 06:39:27.277109 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:27.277161 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260806 06:39:27.346479 21795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:27.346218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:45745)
0806 06:39:27.346372 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:27.346418 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260806 06:39:27.501202 21796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:27.500944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:42869)
0806 06:39:27.501101 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:27.501140 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
I20260806 06:39:27.911888 21797 raft_consensus.cc:482] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:39:27.912079 21797 raft_consensus.cc:504] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:27.912546 21797 leader_election.cc:288] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525), 63b22c163965490b85a29f556da28967 (127.29.142.131:45201)
Aug 06 06:39:27 dist-test-slave-3kk6 krb5kdc[19631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.131@KRBTEST.COM
Aug 06 06:39:27 dist-test-slave-3kk6 krb5kdc[19631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998366, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.130@KRBTEST.COM
I20260806 06:39:27.924517 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:27.912914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:33763 (local address 127.29.142.131:45201)
0806 06:39:27.913161 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:27.913172 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:27.913231 (+ 59us) server_negotiation.cc:407] Connection header received
0806 06:39:27.913747 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:27.913750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:27.913799 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:27.913952 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:27.915035 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:27.916084 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:27.916875 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:27.917291 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:27.917415 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:27.920057 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:27.920063 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:27.920063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:27.920076 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:27.922283 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:27.922947 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:27.922951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:27.922951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:27.923079 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:27.923459 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:27.923460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:27.923460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:27.923783 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:27.923879 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:27.924250 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 06:39:27.924397 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":70,"threads_started":1}
I20260806 06:39:27.924517 21803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:27.912931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:57853 (local address 127.29.142.130:34525)
0806 06:39:27.913107 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:27.913126 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:27.913156 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:39:27.913631 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:27.913633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:27.913676 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:27.913894 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:27.914746 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:27.915785 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:27.916876 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:27.917287 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:27.917413 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:27.920468 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:27.920471 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:27.920471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:27.920480 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:27.922478 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:27.922973 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:27.922976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:27.922976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:27.923079 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:27.923447 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:27.923448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:27.923449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:27.923784 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:27.923876 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:27.924249 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 06:39:27.924397 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":60,"threads_started":1}
I20260806 06:39:27.924544 21799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:27.912769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:34525 (local address 127.29.142.129:57853)
0806 06:39:27.912904 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:27.912917 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:27.912985 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:27.913935 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:27.913938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:27.914342 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:27.914569 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:27.914576 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:27.915894 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:27.915895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:27.916775 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:27.916778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:27.917383 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:27.917384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:27.917516 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:27.918165 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:27.918175 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:27.920332 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:27.922563 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:27.922564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:27.922564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:27.922895 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:27.923178 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:27.923178 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:27.923179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:27.923361 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:27.923879 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:27.923887 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:27.924052 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 06:39:27.924170 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 06:39:27.924385 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":11,"thread_start_us":74,"threads_started":1}
I20260806 06:39:27.924562 21801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:27.912769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:45201 (local address 127.29.142.129:33763)
0806 06:39:27.912932 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:27.912946 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:27.913028 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:27.913968 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:27.913969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:27.914343 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:27.914866 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:27.914869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:27.916182 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:27.916182 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:27.916777 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:27.916779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:27.917390 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:27.917391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:27.917517 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:27.918201 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:27.918205 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:27.919911 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:27.922369 (+ 2458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:27.922375 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:27.922377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:27.922845 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:27.923180 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:27.923180 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:27.923181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:27.923361 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:27.923879 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:27.923887 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:27.924052 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 06:39:27.924170 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 06:39:27.924444 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":141,"thread_start_us":55,"threads_started":1}
W20260806 06:39:27.925458 21265 leader_election.cc:341] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525): Not found: Tablet not found: 1890ebd36edc4f4abfb76140cce0b02b
W20260806 06:39:27.926026 21268 leader_election.cc:341] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63b22c163965490b85a29f556da28967 (127.29.142.131:45201): Not found: Tablet not found: 1890ebd36edc4f4abfb76140cce0b02b
I20260806 06:39:27.926110 21268 leader_election.cc:302] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee6b01b560b144b0a5d1f576796184db; no voters: 63b22c163965490b85a29f556da28967, b9ec3df106514e309c5d903cc1536618
I20260806 06:39:27.926262 21797 raft_consensus.cc:2726] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:39:28.277937 21799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:28.277759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:60745)
0806 06:39:28.277849 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:28.277892 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 06:39:28.347414 21804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:28.347105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:46353)
0806 06:39:28.347284 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:28.347356 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":80,"threads_started":1}
I20260806 06:39:28.501914 21807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:28.501698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:34429)
0806 06:39:28.501832 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:28.501870 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260806 06:39:29.278875 21809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:29.278558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:39113)
0806 06:39:29.278764 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:29.278813 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
I20260806 06:39:29.348341 21810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:29.348069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:48071)
0806 06:39:29.348218 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:29.348289 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260806 06:39:29.502858 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:29.502537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:48661)
0806 06:39:29.502751 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:29.502801 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":76,"threads_started":1}
I20260806 06:39:29.980619 21812 raft_consensus.cc:482] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:39:29.980759 21812 raft_consensus.cc:504] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:29.981014 21812 leader_election.cc:288] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525), 63b22c163965490b85a29f556da28967 (127.29.142.131:45201)
W20260806 06:39:29.981510 21265 leader_election.cc:341] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525): Not found: Tablet not found: 1890ebd36edc4f4abfb76140cce0b02b
W20260806 06:39:29.981619 21268 leader_election.cc:341] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63b22c163965490b85a29f556da28967 (127.29.142.131:45201): Not found: Tablet not found: 1890ebd36edc4f4abfb76140cce0b02b
I20260806 06:39:29.981685 21268 leader_election.cc:302] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee6b01b560b144b0a5d1f576796184db; no voters: 63b22c163965490b85a29f556da28967, b9ec3df106514e309c5d903cc1536618
I20260806 06:39:29.981756 21812 raft_consensus.cc:2726] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:39:30.279734 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:30.279456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:52759)
0806 06:39:30.279612 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:30.279656 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260806 06:39:30.349267 21814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:30.348950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:34491)
0806 06:39:30.349109 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:30.349170 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260806 06:39:30.503661 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:30.503422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:42779)
0806 06:39:30.503564 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:30.503611 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":67,"threads_started":1}
I20260806 06:39:31.280592 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:31.280316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:57269)
0806 06:39:31.280497 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:31.280542 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":116,"threads_started":1}
I20260806 06:39:31.350160 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:31.349895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:34299)
0806 06:39:31.350071 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:31.350111 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":119,"threads_started":1}
I20260806 06:39:31.504575 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:31.504246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:32887)
0806 06:39:31.504476 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:31.504524 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":138,"threads_started":1}
I20260806 06:39:31.607383 21822 raft_consensus.cc:482] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:39:31.607532 21822 raft_consensus.cc:504] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:31.607774 21822 leader_election.cc:288] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525), 63b22c163965490b85a29f556da28967 (127.29.142.131:45201)
W20260806 06:39:31.608215 21268 leader_election.cc:341] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63b22c163965490b85a29f556da28967 (127.29.142.131:45201): Not found: Tablet not found: 1890ebd36edc4f4abfb76140cce0b02b
W20260806 06:39:31.608316 21265 leader_election.cc:341] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525): Not found: Tablet not found: 1890ebd36edc4f4abfb76140cce0b02b
I20260806 06:39:31.608413 21265 leader_election.cc:302] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee6b01b560b144b0a5d1f576796184db; no voters: 63b22c163965490b85a29f556da28967, b9ec3df106514e309c5d903cc1536618
I20260806 06:39:31.608482 21822 raft_consensus.cc:2726] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:39:32.281571 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:32.281286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:54257)
0806 06:39:32.281466 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:32.281520 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
I20260806 06:39:32.351086 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:32.350835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:56955)
0806 06:39:32.350984 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:32.351036 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260806 06:39:32.505414 21826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:32.505124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:44057)
0806 06:39:32.505281 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:32.505352 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260806 06:39:33.282459 21828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:33.282184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:48179)
0806 06:39:33.282342 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:33.282389 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260806 06:39:33.351788 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:33.351539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:58447)
0806 06:39:33.351696 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:33.351735 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260806 06:39:33.506217 21830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:33.505969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:34807)
0806 06:39:33.506120 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:33.506168 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260806 06:39:34.283281 21831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:34.283025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:46835)
0806 06:39:34.283179 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:34.283232 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260806 06:39:34.352572 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:34.352289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:45591)
0806 06:39:34.352475 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:34.352522 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":83,"threads_started":1}
I20260806 06:39:34.403043 21834 raft_consensus.cc:482] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:39:34.403147 21834 raft_consensus.cc:504] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:34.403326 21834 leader_election.cc:288] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525), 63b22c163965490b85a29f556da28967 (127.29.142.131:45201)
W20260806 06:39:34.403798 21265 leader_election.cc:341] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525): Not found: Tablet not found: 1890ebd36edc4f4abfb76140cce0b02b
W20260806 06:39:34.403878 21268 leader_election.cc:341] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63b22c163965490b85a29f556da28967 (127.29.142.131:45201): Not found: Tablet not found: 1890ebd36edc4f4abfb76140cce0b02b
I20260806 06:39:34.403960 21268 leader_election.cc:302] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee6b01b560b144b0a5d1f576796184db; no voters: 63b22c163965490b85a29f556da28967, b9ec3df106514e309c5d903cc1536618
I20260806 06:39:34.404049 21834 raft_consensus.cc:2726] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:39:34.507171 21835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:34.506902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:33585)
0806 06:39:34.507063 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:34.507104 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260806 06:39:35.284096 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:35.283839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:38553)
0806 06:39:35.284008 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:35.284043 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
I20260806 06:39:35.353397 21837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:35.353139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:58615)
0806 06:39:35.353288 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:35.353338 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260806 06:39:35.507879 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:35.507618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:49567)
0806 06:39:35.507782 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:35.507828 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":67,"threads_started":1}
I20260806 06:39:36.284826 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:36.284611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:55461)
0806 06:39:36.284738 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:36.284776 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260806 06:39:36.354197 21840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:36.353948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:57281)
0806 06:39:36.354099 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:36.354148 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260806 06:39:36.508625 21841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:36.508389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:33133)
0806 06:39:36.508532 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:36.508575 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260806 06:39:37.285722 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:37.285412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:60897)
0806 06:39:37.285614 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:37.285663 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":137,"threads_started":1}
I20260806 06:39:37.355181 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:37.354866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:33649)
0806 06:39:37.355063 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:37.355119 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":104,"threads_started":1}
I20260806 06:39:37.509546 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:37.509254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:32791)
0806 06:39:37.509439 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:37.509494 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":91,"threads_started":1}
I20260806 06:39:38.286686 21846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:38.286384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:52491)
0806 06:39:38.286551 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:38.286609 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260806 06:39:38.356019 21847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:38.355770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:50609)
0806 06:39:38.355931 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:38.355966 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260806 06:39:38.510452 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:38.510184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:34107)
0806 06:39:38.510345 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:38.510397 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
I20260806 06:39:39.194869 21849 raft_consensus.cc:482] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:39:39.194986 21849 raft_consensus.cc:504] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ec3df106514e309c5d903cc1536618" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 34525 } } peers { permanent_uuid: "63b22c163965490b85a29f556da28967" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 45201 } } peers { permanent_uuid: "ee6b01b560b144b0a5d1f576796184db" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42987 } }
I20260806 06:39:39.195158 21849 leader_election.cc:288] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525), 63b22c163965490b85a29f556da28967 (127.29.142.131:45201)
W20260806 06:39:39.195736 21268 leader_election.cc:341] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63b22c163965490b85a29f556da28967 (127.29.142.131:45201): Not found: Tablet not found: 1890ebd36edc4f4abfb76140cce0b02b
W20260806 06:39:39.195829 21265 leader_election.cc:341] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9ec3df106514e309c5d903cc1536618 (127.29.142.130:34525): Not found: Tablet not found: 1890ebd36edc4f4abfb76140cce0b02b
I20260806 06:39:39.195878 21265 leader_election.cc:302] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee6b01b560b144b0a5d1f576796184db; no voters: 63b22c163965490b85a29f556da28967, b9ec3df106514e309c5d903cc1536618
I20260806 06:39:39.195981 21849 raft_consensus.cc:2726] T 1890ebd36edc4f4abfb76140cce0b02b P ee6b01b560b144b0a5d1f576796184db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:39:39.287516 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:39.287268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:48839)
0806 06:39:39.287413 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:39.287463 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260806 06:39:39.356920 21853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:39.356567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:55119)
0806 06:39:39.356808 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:39.356869 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":93,"threads_started":1}
I20260806 06:39:39.511461 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:39.511156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:54719)
0806 06:39:39.511358 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:39.511409 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":101,"threads_started":1}
I20260806 06:39:40.288429 21856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:40.288147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:38069)
0806 06:39:40.288324 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:40.288378 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":85,"threads_started":1}
I20260806 06:39:40.357632 21858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:40.357405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:45921)
0806 06:39:40.357538 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:40.357583 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260806 06:39:40.512387 21859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:40.512094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:45539)
0806 06:39:40.512275 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:40.512334 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
I20260806 06:39:41.289417 21862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:41.289125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.129:48625)
0806 06:39:41.289304 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:41.289363 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":81,"threads_started":1}
I20260806 06:39:41.358785 21863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:41.358438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.130:45523)
0806 06:39:41.358598 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:41.358724 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260806 06:39:41.508153 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 21199
I20260806 06:39:41.514130 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:41.513162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:43607 (local address 127.29.142.131:55419)
0806 06:39:41.513993 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:41.514034 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.190:43607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":755,"threads_started":1}
I20260806 06:39:41.518348 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 21426
I20260806 06:39:41.529712 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 21614
2026-08-06T06:39:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58788 ms)
[----------] 30 tests from SecurityITest (243770 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:39:41 dist-test-slave-3kk6 krb5kdc[21869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:39:41 dist-test-slave-3kk6 krb5kdc[21869](info): set up 2 sockets
Aug 06 06:39:41 dist-test-slave-3kk6 krb5kdc[21869](info): commencing operation
krb5kdc: starting...
W20260806 06:39:42.765466 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:39:42 dist-test-slave-3kk6 krb5kdc[21869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:39:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:39:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:39:42.895780 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:41389
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:45991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:41389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:42.981802 21888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:42.982079 21888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:42.982120 21888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:42.983369 21888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:42.983440 21888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:42.983465 21888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:42.983489 21888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:42.983520 21888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:42.985044 21888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:45991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:41389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:41389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:42.985349 21888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:42.985595 21888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:39:42.991176 21888 server_base.cc:1034] running on GCE node
W20260806 06:39:42.991144 21896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:42.991144 21900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:42.991137 21897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:42.991626 21888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:42.991927 21888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:42.993083 21888 hybrid_clock.cc:648] HybridClock initialized: now 1785998382993060 us; error 45 us; skew 500 ppm
Aug 06 06:39:42 dist-test-slave-3kk6 krb5kdc[21869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998382, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:43.000078 21888 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:39:43.000622 21888 webserver.cc:460] Webserver started at http://127.29.142.190:34523/ using document root <none> and password file <none>
I20260806 06:39:43.000891 21888 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:43.000963 21888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:43.001119 21888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:43.002236 21888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "102a26ce95314edf865422103bdeb4f4"
format_stamp: "Formatted at 2026-08-06 06:39:42 on dist-test-slave-3kk6"
server_key: "1be1ab99443efb9f815e46bfa82dad1c"
server_key_iv: "83a6accd005ccfbc812c3d845fb844d0"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.002622 21888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "102a26ce95314edf865422103bdeb4f4"
format_stamp: "Formatted at 2026-08-06 06:39:42 on dist-test-slave-3kk6"
server_key: "1be1ab99443efb9f815e46bfa82dad1c"
server_key_iv: "83a6accd005ccfbc812c3d845fb844d0"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.004086 21888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:43.004869 21908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.005072 21888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:43.005188 21888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "102a26ce95314edf865422103bdeb4f4"
format_stamp: "Formatted at 2026-08-06 06:39:42 on dist-test-slave-3kk6"
server_key: "1be1ab99443efb9f815e46bfa82dad1c"
server_key_iv: "83a6accd005ccfbc812c3d845fb844d0"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.005267 21888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:43.017642 21888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:43.019948 21888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:43.020121 21888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:43.024530 21888 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:41389
I20260806 06:39:43.024648 21978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:41389 every 8 connection(s)
I20260806 06:39:43.024983 21888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:43.025895 21979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:43.028379 21979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4: Bootstrap starting.
I20260806 06:39:43.029009 21979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:43.029318 21979 log.cc:824] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:43.030130 21979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4: No bootstrap required, opened a new log
I20260806 06:39:43.031836 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 21888
I20260806 06:39:43.031948 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:39:43.032147 30266 external_mini_cluster.cc:1428] Setting key 31cb81b36e14d1b5ab746c9582078736
I20260806 06:39:43.032344 21979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102a26ce95314edf865422103bdeb4f4" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41389 } }
I20260806 06:39:43.032524 21979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:43.032591 21979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 102a26ce95314edf865422103bdeb4f4, State: Initialized, Role: FOLLOWER
I20260806 06:39:43.032733 21979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102a26ce95314edf865422103bdeb4f4" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41389 } }
I20260806 06:39:43.032836 21979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:43.032872 21979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:43.032958 21979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:43.034098 21979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102a26ce95314edf865422103bdeb4f4" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41389 } }
I20260806 06:39:43.034209 21979 leader_election.cc:302] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 102a26ce95314edf865422103bdeb4f4; no voters:
I20260806 06:39:43.034373 21979 leader_election.cc:288] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:39:43.034451 21984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:43.034637 21979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:43.034667 21984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [term 1 LEADER]: Becoming Leader. State: Replica: 102a26ce95314edf865422103bdeb4f4, State: Running, Role: LEADER
I20260806 06:39:43.034905 21984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102a26ce95314edf865422103bdeb4f4" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41389 } }
I20260806 06:39:43.035318 21985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "102a26ce95314edf865422103bdeb4f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102a26ce95314edf865422103bdeb4f4" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41389 } } }
I20260806 06:39:43.035413 21985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:43.035581 21986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 102a26ce95314edf865422103bdeb4f4. Latest consensus state: current_term: 1 leader_uuid: "102a26ce95314edf865422103bdeb4f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102a26ce95314edf865422103bdeb4f4" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41389 } } }
I20260806 06:39:43.035640 21986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4 [sys.catalog]: This master's current role is: LEADER
W20260806 06:39:43.036506 22002 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:39:43.036598 22002 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:39:43.036687 21994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:43.036888 21994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:39:43.038501 21994 catalog_manager.cc:1347] Generated new cluster ID: 03b39747d77b4b35863dd32ea6bec523
I20260806 06:39:43.038565 21994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:39:43 dist-test-slave-3kk6 krb5kdc[21869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:43.044325 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.032727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46128 (local address 127.29.142.190:41389)
0806 06:39:43.032952 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.032961 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.032999 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:39:43.033135 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.033143 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.033200 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.033373 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:43.033974 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.035400 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.036562 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.037175 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.037307 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.040266 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:43.040270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:43.040271 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:43.040283 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:43.042380 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:43.042836 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:43.042840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:43.042840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:43.042966 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:43.043229 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:43.043230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:43.043230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:43.043559 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:43.043730 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.043834 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.044096 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":105,"threads_started":1}
I20260806 06:39:43.044891 21994 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:39:43.045415 21994 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:43.050516 21994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4: Generated new TSK 0
I20260806 06:39:43.050714 21994 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:39:43.081758 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41389
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:45991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:43.169106 22009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:43.169355 22009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:43.169382 22009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:43.170513 22009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:43.170552 22009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:43.170578 22009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:39:43.172176 22009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:45991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:43.172431 22009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:43.172672 22009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:43.177896 22018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:43.177910 22022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:43.177910 22019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:43.178325 22009 server_base.cc:1034] running on GCE node
I20260806 06:39:43.178526 22009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:43.178799 22009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:43.179968 22009 hybrid_clock.cc:648] HybridClock initialized: now 1785998383179950 us; error 34 us; skew 500 ppm
Aug 06 06:39:43 dist-test-slave-3kk6 krb5kdc[21869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998383, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:43.182802 22009 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:39:43.183243 22009 webserver.cc:460] Webserver started at http://127.29.142.129:41151/ using document root <none> and password file <none>
I20260806 06:39:43.183501 22009 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:43.183573 22009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:43.183733 22009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:43.184857 22009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "14df8fa233694ec5ad1e233001c9631e"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "1042bba017d57602af18b05b48cbc8c8"
server_key_iv: "94f22db9dd494a34700bef96582fef0d"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.185262 22009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "14df8fa233694ec5ad1e233001c9631e"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "1042bba017d57602af18b05b48cbc8c8"
server_key_iv: "94f22db9dd494a34700bef96582fef0d"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.186787 22009 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:39:43.187569 22029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.187749 22009 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:39:43.187858 22009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "14df8fa233694ec5ad1e233001c9631e"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "1042bba017d57602af18b05b48cbc8c8"
server_key_iv: "94f22db9dd494a34700bef96582fef0d"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.187937 22009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:43.200788 22009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:43.202991 22009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:43.203164 22009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:43.203418 22009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:43.203766 22009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:43.203830 22009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.203876 22009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:43.203905 22009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.210055 22009 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:42961
I20260806 06:39:43.210134 22189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:42961 every 8 connection(s)
I20260806 06:39:43.210495 22009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:39:43 dist-test-slave-3kk6 krb5kdc[21869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998383, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:43.217898 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 22009
I20260806 06:39:43.218024 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:39:43.218258 30266 external_mini_cluster.cc:1428] Setting key 3a68918a3dff5c2885329a7162e1e2e2
I20260806 06:39:43.220839 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.211347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:57095 (local address 127.29.142.190:41389)
0806 06:39:43.211438 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.211441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.211624 (+ 183us) server_negotiation.cc:407] Connection header received
0806 06:39:43.212133 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.212134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.212144 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.212197 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:43.213218 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.213730 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.214786 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.215000 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.215065 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.217396 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:43.217398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:43.217398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:43.217405 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:43.219201 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:43.219692 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:43.219694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:43.219694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:43.219766 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:43.220069 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:43.220071 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:43.220072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:43.220203 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:43.220289 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.220597 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.220703 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:39:43.220986 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.211303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41389 (local address 127.29.142.129:57095)
0806 06:39:43.211486 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:43.211524 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:43.212038 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:43.212259 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:43.212262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:43.212886 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:43.213092 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.213098 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.213845 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.213846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.214674 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.214676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.215084 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.215085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.215188 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.215819 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:43.215828 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:43.217304 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:43.219323 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:43.219333 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:43.219336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:43.219589 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:43.219850 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:43.219851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:43.219851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:43.219976 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:43.220278 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:43.220287 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:43.220444 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 06:39:43.220558 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:39:43.220737 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260806 06:39:43.221483 22190 heartbeater.cc:347] Connected to a master server at 127.29.142.190:41389
I20260806 06:39:43.221582 22190 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:43.221771 22190 heartbeater.cc:511] Master 127.29.142.190:41389 requested a full tablet report, sending...
I20260806 06:39:43.222337 21930 ts_manager.cc:194] Registered new tserver with Master: 14df8fa233694ec5ad1e233001c9631e (127.29.142.129:42961)
I20260806 06:39:43.223079 21930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:57095
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:39:43.246312 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41389
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:45991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:43.333475 22195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:43.333716 22195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:43.333742 22195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:43.335155 22195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:43.335210 22195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:43.335239 22195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:39:43.336935 22195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:45991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:43.337210 22195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:43.337477 22195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:43.342957 22203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:43.342952 22204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:43.342953 22207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:43.343278 22195 server_base.cc:1034] running on GCE node
I20260806 06:39:43.343492 22195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:43.343755 22195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:43.344924 22195 hybrid_clock.cc:648] HybridClock initialized: now 1785998383344915 us; error 36 us; skew 500 ppm
Aug 06 06:39:43 dist-test-slave-3kk6 krb5kdc[21869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998383, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:43.347707 22195 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:39:43.348215 22195 webserver.cc:460] Webserver started at http://127.29.142.130:42419/ using document root <none> and password file <none>
I20260806 06:39:43.348483 22195 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:43.348556 22195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:43.348713 22195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:43.349843 22195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "8418d90ccc7142f4aa467a9b86a9f860"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "0895bdaa62de62ef6a10d5ef694229e3"
server_key_iv: "6aef7d776f034864cd8243f5bc50cfcf"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.350263 22195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "8418d90ccc7142f4aa467a9b86a9f860"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "0895bdaa62de62ef6a10d5ef694229e3"
server_key_iv: "6aef7d776f034864cd8243f5bc50cfcf"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.351852 22195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:39:43.352907 22219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.353116 22195 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:39:43.353232 22195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "8418d90ccc7142f4aa467a9b86a9f860"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "0895bdaa62de62ef6a10d5ef694229e3"
server_key_iv: "6aef7d776f034864cd8243f5bc50cfcf"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.353317 22195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:43.361378 22195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:43.363906 22195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:43.364091 22195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:43.364347 22195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:43.364673 22195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:43.364732 22195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.364804 22195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:43.364835 22195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.370926 22195 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:38843
I20260806 06:39:43.371026 22374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:38843 every 8 connection(s)
I20260806 06:39:43.371622 22195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:39:43.372110 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 22195
I20260806 06:39:43.372231 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:39:43.372464 30266 external_mini_cluster.cc:1428] Setting key 22bf978048f448c5403affc5436803c9
Aug 06 06:39:43 dist-test-slave-3kk6 krb5kdc[21869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998383, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
I20260806 06:39:43.383893 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.372650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:44561 (local address 127.29.142.190:41389)
0806 06:39:43.372717 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.372719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.373016 (+ 297us) server_negotiation.cc:407] Connection header received
0806 06:39:43.373718 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.373720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.373733 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.373775 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:43.375274 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.375780 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.376814 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.377018 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.377106 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.380374 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:43.380376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:43.380377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:43.380387 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:43.382255 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:43.382750 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:43.382751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:43.382751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:43.382816 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:43.383084 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:43.383085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:43.383085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:43.383245 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:43.383342 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.383672 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.383794 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:39:43.384049 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.372561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41389 (local address 127.29.142.130:44561)
0806 06:39:43.372797 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:43.372898 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:43.373607 (+ 709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:43.373962 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:43.373966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:43.374903 (+ 937us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:43.375145 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.375151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.375880 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.375881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.376692 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.376695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.377118 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.377119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.377292 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.378836 (+ 1544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:43.378845 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:43.380267 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:43.382357 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:43.382365 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:43.382367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:43.382602 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:43.382890 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:43.382891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:43.382892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:43.383004 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:43.383332 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:43.383340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:43.383510 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 06:39:43.383633 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 06:39:43.383811 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
I20260806 06:39:43.384644 22375 heartbeater.cc:347] Connected to a master server at 127.29.142.190:41389
I20260806 06:39:43.384773 22375 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:43.384967 22375 heartbeater.cc:511] Master 127.29.142.190:41389 requested a full tablet report, sending...
I20260806 06:39:43.385404 21930 ts_manager.cc:194] Registered new tserver with Master: 8418d90ccc7142f4aa467a9b86a9f860 (127.29.142.130:38843)
I20260806 06:39:43.385879 21930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:44561
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:39:43.402120 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41389
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:45991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:43.489037 22380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:43.489277 22380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:43.489303 22380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:43.490636 22380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:43.490731 22380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:43.490761 22380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:39:43.492404 22380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:45991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:41389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:43.492669 22380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:43.492928 22380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:43.498572 22387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:43.498489 22389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:43.498489 22386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:43.498847 22380 server_base.cc:1034] running on GCE node
I20260806 06:39:43.499116 22380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:43.499434 22380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:43.500604 22380 hybrid_clock.cc:648] HybridClock initialized: now 1785998383500582 us; error 50 us; skew 500 ppm
Aug 06 06:39:43 dist-test-slave-3kk6 krb5kdc[21869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998383, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:43.503454 22380 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:39:43.503921 22380 webserver.cc:460] Webserver started at http://127.29.142.131:37007/ using document root <none> and password file <none>
I20260806 06:39:43.504122 22380 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:43.504168 22380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:43.504277 22380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:43.505457 22380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "5e5b96e52b4142d5b6caee27a681b79d"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "6eda7d02a93739d4156670d9d4fe7e20"
server_key_iv: "0a4f8661ead1b5ae4d9b9c6a2a9823f3"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.505808 22380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "5e5b96e52b4142d5b6caee27a681b79d"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "6eda7d02a93739d4156670d9d4fe7e20"
server_key_iv: "0a4f8661ead1b5ae4d9b9c6a2a9823f3"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.507330 22380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:39:43.508365 22398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.508586 22380 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:43.508687 22380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "5e5b96e52b4142d5b6caee27a681b79d"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "6eda7d02a93739d4156670d9d4fe7e20"
server_key_iv: "0a4f8661ead1b5ae4d9b9c6a2a9823f3"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.508773 22380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:43.516806 22380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:43.519328 22380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:43.519495 22380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:43.519767 22380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:43.520120 22380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:43.520176 22380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.520212 22380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:43.520241 22380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.526322 22380 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:33217
I20260806 06:39:43.526405 22558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:33217 every 8 connection(s)
I20260806 06:39:43.526804 22380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:39:43.528025 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 22380
I20260806 06:39:43.528151 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:39:43.528359 30266 external_mini_cluster.cc:1428] Setting key 44f05728831d13fe3f4c5af3fed4540a
Aug 06 06:39:43 dist-test-slave-3kk6 krb5kdc[21869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998383, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:43.537575 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.527687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:54057 (local address 127.29.142.190:41389)
0806 06:39:43.527760 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.527763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.527994 (+ 231us) server_negotiation.cc:407] Connection header received
0806 06:39:43.528749 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.528751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.528765 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.528809 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:43.530202 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.530802 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.531839 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.532124 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.532198 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.534600 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:43.534601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:43.534602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:43.534609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:43.536239 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:43.536701 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:43.536701 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:43.536702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:43.536779 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:43.537020 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:43.537021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:43.537021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:43.537129 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:43.537180 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.537432 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.537503 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:39:43.537758 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.527650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:41389 (local address 127.29.142.131:54057)
0806 06:39:43.527864 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:43.527902 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:43.528639 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:43.528896 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:43.528901 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:43.529805 (+ 904us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:43.530095 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.530102 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.530899 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.530900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.531742 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.531745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.532211 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.532212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.532356 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.532907 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:43.532917 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:43.534497 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:43.536360 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:43.536368 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:43.536370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:43.536629 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:43.536841 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:43.536842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:43.536842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:43.536961 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:43.537186 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:43.537190 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:43.537316 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 06:39:43.537403 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:39:43.537535 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":132,"threads_started":1}
I20260806 06:39:43.538333 22559 heartbeater.cc:347] Connected to a master server at 127.29.142.190:41389
I20260806 06:39:43.538461 22559 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:43.538714 22559 heartbeater.cc:511] Master 127.29.142.190:41389 requested a full tablet report, sending...
I20260806 06:39:43.539184 21930 ts_manager.cc:194] Registered new tserver with Master: 5e5b96e52b4142d5b6caee27a681b79d (127.29.142.131:33217)
I20260806 06:39:43.539609 21930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:54057
I20260806 06:39:43.540536 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:39:43.548736 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.542019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46132 (local address 127.29.142.190:41389)
0806 06:39:43.542088 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.542090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.542636 (+ 546us) server_negotiation.cc:407] Connection header received
0806 06:39:43.542736 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.542737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.542748 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.542798 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:43.543397 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.543889 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.544656 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.544864 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.544936 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.546168 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:43.546169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:43.546169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:43.546177 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:43.547484 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:43.547879 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:43.547880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:43.547881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:43.547944 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:43.548155 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:43.548156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:43.548157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:43.548318 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:43.548425 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.548529 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.548618 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:39:43.550263 21930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:39:43.551946 21930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:39:43.559142 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.554731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46108 (local address 127.29.142.129:42961)
0806 06:39:43.554917 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.554923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.554935 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:39:43.554980 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.554982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.555001 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.555333 (+ 332us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:43.555408 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.556571 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.557849 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.558618 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.558770 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.558843 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.558994 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.559050 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":108,"threads_started":1}
I20260806 06:39:43.559128 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.554368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42961 (local address 127.0.0.1:46108)
0806 06:39:43.554533 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:43.554578 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:43.554647 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:43.555126 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:43.555130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:43.555132 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:43.555247 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.555250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.556666 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.556667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.557753 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.557757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.558782 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.558783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.558871 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.558888 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 06:39:43.558965 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:39:43.559004 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":70,"thread_start_us":54,"threads_started":1}
I20260806 06:39:43.559338 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.554925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44826 (local address 127.29.142.131:33217)
0806 06:39:43.555168 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.555173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.555184 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:39:43.555231 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.555233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.555250 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.555327 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:43.555535 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.556294 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.558012 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.558749 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.558811 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.558867 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.559194 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.559242 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":114,"threads_started":1}
I20260806 06:39:43.559145 22576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.554372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:38843 (local address 127.0.0.1:46894)
0806 06:39:43.554568 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:43.554580 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:43.554671 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:43.555142 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:43.555143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:43.555145 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:43.555247 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.555250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.556987 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.556987 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.558084 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.558086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.558827 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.558828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.558879 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.558889 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 06:39:43.558939 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:39:43.559002 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":28,"thread_start_us":149,"threads_started":1}
I20260806 06:39:43.559386 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.554496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46894 (local address 127.29.142.130:38843)
0806 06:39:43.554971 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.554975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.554984 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:39:43.555021 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.555024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.555040 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.555317 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:43.555407 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.556559 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.558166 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.558718 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.559146 (+ 428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.559209 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.559240 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.559288 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":90,"threads_started":1}
I20260806 06:39:43.559751 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.554687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:33217 (local address 127.0.0.1:44826)
0806 06:39:43.554800 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:43.554811 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:43.554857 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:43.555399 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:43.555400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:43.555403 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:43.555459 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.555460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.556432 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.556433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.557912 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.557916 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.558902 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.558903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.558992 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.558997 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:39:43.559050 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:39:43.559094 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":25,"thread_start_us":88,"threads_started":1}
I20260806 06:39:43.560153 22279 tablet_service.cc:1467] Processing CreateTablet for tablet c09ac8885f7d42cb9d837d3d0d5352b5 (DEFAULT_TABLE table=test-table [id=12eae8c63ab14d969d8b8b03331858e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:43.560388 22097 tablet_service.cc:1467] Processing CreateTablet for tablet c09ac8885f7d42cb9d837d3d0d5352b5 (DEFAULT_TABLE table=test-table [id=12eae8c63ab14d969d8b8b03331858e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:43.560586 22279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c09ac8885f7d42cb9d837d3d0d5352b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:43.560637 22464 tablet_service.cc:1467] Processing CreateTablet for tablet c09ac8885f7d42cb9d837d3d0d5352b5 (DEFAULT_TABLE table=test-table [id=12eae8c63ab14d969d8b8b03331858e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:43.560777 22097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c09ac8885f7d42cb9d837d3d0d5352b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:43.560971 22464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c09ac8885f7d42cb9d837d3d0d5352b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:43.563820 22588 tablet_bootstrap.cc:492] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e: Bootstrap starting.
I20260806 06:39:43.564328 22589 tablet_bootstrap.cc:492] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860: Bootstrap starting.
I20260806 06:39:43.564597 22590 tablet_bootstrap.cc:492] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d: Bootstrap starting.
I20260806 06:39:43.564848 22588 tablet_bootstrap.cc:654] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:43.565026 22589 tablet_bootstrap.cc:654] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:43.565279 22590 tablet_bootstrap.cc:654] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:43.565659 22588 log.cc:824] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:43.565667 22589 log.cc:824] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:43.565872 22590 log.cc:824] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:43.566601 22588 tablet_bootstrap.cc:492] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e: No bootstrap required, opened a new log
I20260806 06:39:43.566766 22588 ts_tablet_manager.cc:1397] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:39:43.566840 22590 tablet_bootstrap.cc:492] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d: No bootstrap required, opened a new log
I20260806 06:39:43.566852 22589 tablet_bootstrap.cc:492] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860: No bootstrap required, opened a new log
I20260806 06:39:43.566915 22589 ts_tablet_manager.cc:1397] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:39:43.566918 22590 ts_tablet_manager.cc:1397] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:39:43.568665 22590 raft_consensus.cc:348] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8418d90ccc7142f4aa467a9b86a9f860" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38843 } } peers { permanent_uuid: "14df8fa233694ec5ad1e233001c9631e" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42961 } } peers { permanent_uuid: "5e5b96e52b4142d5b6caee27a681b79d" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 33217 } }
I20260806 06:39:43.568827 22590 raft_consensus.cc:374] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:43.568852 22590 raft_consensus.cc:729] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5b96e52b4142d5b6caee27a681b79d, State: Initialized, Role: FOLLOWER
I20260806 06:39:43.569003 22590 consensus_queue.cc:260] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8418d90ccc7142f4aa467a9b86a9f860" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38843 } } peers { permanent_uuid: "14df8fa233694ec5ad1e233001c9631e" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42961 } } peers { permanent_uuid: "5e5b96e52b4142d5b6caee27a681b79d" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 33217 } }
I20260806 06:39:43.568977 22589 raft_consensus.cc:348] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8418d90ccc7142f4aa467a9b86a9f860" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38843 } } peers { permanent_uuid: "14df8fa233694ec5ad1e233001c9631e" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42961 } } peers { permanent_uuid: "5e5b96e52b4142d5b6caee27a681b79d" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 33217 } }
I20260806 06:39:43.569169 22589 raft_consensus.cc:374] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:43.569213 22589 raft_consensus.cc:729] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8418d90ccc7142f4aa467a9b86a9f860, State: Initialized, Role: FOLLOWER
I20260806 06:39:43.569329 22590 ts_tablet_manager.cc:1428] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:39:43.569346 22589 consensus_queue.cc:260] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8418d90ccc7142f4aa467a9b86a9f860" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38843 } } peers { permanent_uuid: "14df8fa233694ec5ad1e233001c9631e" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42961 } } peers { permanent_uuid: "5e5b96e52b4142d5b6caee27a681b79d" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 33217 } }
I20260806 06:39:43.569487 22559 heartbeater.cc:503] Master 127.29.142.190:41389 was elected leader, sending a full tablet report...
I20260806 06:39:43.569628 22589 ts_tablet_manager.cc:1428] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:39:43.569780 22375 heartbeater.cc:503] Master 127.29.142.190:41389 was elected leader, sending a full tablet report...
I20260806 06:39:43.569653 22588 raft_consensus.cc:348] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8418d90ccc7142f4aa467a9b86a9f860" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38843 } } peers { permanent_uuid: "14df8fa233694ec5ad1e233001c9631e" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42961 } } peers { permanent_uuid: "5e5b96e52b4142d5b6caee27a681b79d" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 33217 } }
I20260806 06:39:43.569823 22588 raft_consensus.cc:374] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:43.569871 22588 raft_consensus.cc:729] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14df8fa233694ec5ad1e233001c9631e, State: Initialized, Role: FOLLOWER
I20260806 06:39:43.569988 22588 consensus_queue.cc:260] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8418d90ccc7142f4aa467a9b86a9f860" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38843 } } peers { permanent_uuid: "14df8fa233694ec5ad1e233001c9631e" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42961 } } peers { permanent_uuid: "5e5b96e52b4142d5b6caee27a681b79d" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 33217 } }
I20260806 06:39:43.570308 22588 ts_tablet_manager.cc:1428] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:39:43.570420 22190 heartbeater.cc:503] Master 127.29.142.190:41389 was elected leader, sending a full tablet report...
I20260806 06:39:43.587373 22598 raft_consensus.cc:482] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:39:43.587558 22598 raft_consensus.cc:504] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8418d90ccc7142f4aa467a9b86a9f860" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38843 } } peers { permanent_uuid: "14df8fa233694ec5ad1e233001c9631e" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42961 } } peers { permanent_uuid: "5e5b96e52b4142d5b6caee27a681b79d" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 33217 } }
I20260806 06:39:43.588016 22598 leader_election.cc:288] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8418d90ccc7142f4aa467a9b86a9f860 (127.29.142.130:38843), 5e5b96e52b4142d5b6caee27a681b79d (127.29.142.131:33217)
I20260806 06:39:43.591125 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.588322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:34187 (local address 127.29.142.130:38843)
0806 06:39:43.588395 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.588397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.588406 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:39:43.588437 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.588438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.588449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.588523 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:43.588689 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.589174 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.590176 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.590791 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.590850 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.590895 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.591034 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.591066 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:39:43.591148 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.588213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:38843 (local address 127.29.142.129:34187)
0806 06:39:43.588284 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:43.588304 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:43.588377 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:43.588527 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:43.588528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:43.588529 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:43.588618 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.588620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.589269 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.589271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.590045 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.590047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.590865 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.590866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.590955 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.590959 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:39:43.591028 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:39:43.591070 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:39:43.591513 22305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c09ac8885f7d42cb9d837d3d0d5352b5" candidate_uuid: "14df8fa233694ec5ad1e233001c9631e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8418d90ccc7142f4aa467a9b86a9f860" is_pre_election: true
I20260806 06:39:43.591679 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.588380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:33217 (local address 127.29.142.129:41805)
0806 06:39:43.588582 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:43.588592 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:43.588659 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:43.588838 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:43.588839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:43.588841 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:43.588919 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.588922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.589584 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.589585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.590554 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.590557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.591402 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.591403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.591476 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.591481 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:39:43.591551 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:39:43.591589 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":128,"threads_started":1}
I20260806 06:39:43.591711 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.588483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:41805 (local address 127.29.142.131:33217)
0806 06:39:43.588548 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.588550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.588665 (+ 115us) server_negotiation.cc:407] Connection header received
0806 06:39:43.588726 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.588727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.588743 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.588808 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:43.589004 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.589493 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.590703 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.591323 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.591405 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.591414 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.591577 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.591611 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:39:43.591714 22305 raft_consensus.cc:2454] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14df8fa233694ec5ad1e233001c9631e in term 0.
I20260806 06:39:43.592058 22031 leader_election.cc:302] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14df8fa233694ec5ad1e233001c9631e, 8418d90ccc7142f4aa467a9b86a9f860; no voters:
I20260806 06:39:43.592164 22493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c09ac8885f7d42cb9d837d3d0d5352b5" candidate_uuid: "14df8fa233694ec5ad1e233001c9631e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5b96e52b4142d5b6caee27a681b79d" is_pre_election: true
I20260806 06:39:43.592196 22598 raft_consensus.cc:2781] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:39:43.592270 22598 raft_consensus.cc:482] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:39:43.592288 22598 raft_consensus.cc:3037] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:43.592330 22493 raft_consensus.cc:2454] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14df8fa233694ec5ad1e233001c9631e in term 0.
I20260806 06:39:43.593118 22598 raft_consensus.cc:504] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8418d90ccc7142f4aa467a9b86a9f860" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38843 } } peers { permanent_uuid: "14df8fa233694ec5ad1e233001c9631e" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42961 } } peers { permanent_uuid: "5e5b96e52b4142d5b6caee27a681b79d" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 33217 } }
I20260806 06:39:43.593271 22598 leader_election.cc:288] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [CANDIDATE]: Term 1 election: Requested vote from peers 8418d90ccc7142f4aa467a9b86a9f860 (127.29.142.130:38843), 5e5b96e52b4142d5b6caee27a681b79d (127.29.142.131:33217)
I20260806 06:39:43.593464 22305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c09ac8885f7d42cb9d837d3d0d5352b5" candidate_uuid: "14df8fa233694ec5ad1e233001c9631e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8418d90ccc7142f4aa467a9b86a9f860"
I20260806 06:39:43.593478 22493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c09ac8885f7d42cb9d837d3d0d5352b5" candidate_uuid: "14df8fa233694ec5ad1e233001c9631e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5b96e52b4142d5b6caee27a681b79d"
I20260806 06:39:43.593551 22305 raft_consensus.cc:3037] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:43.593561 22493 raft_consensus.cc:3037] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:43.594417 22493 raft_consensus.cc:2454] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14df8fa233694ec5ad1e233001c9631e in term 1.
I20260806 06:39:43.594427 22305 raft_consensus.cc:2454] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14df8fa233694ec5ad1e233001c9631e in term 1.
I20260806 06:39:43.594605 22031 leader_election.cc:302] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14df8fa233694ec5ad1e233001c9631e, 5e5b96e52b4142d5b6caee27a681b79d; no voters:
I20260806 06:39:43.594810 22598 raft_consensus.cc:2781] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:43.594980 22598 raft_consensus.cc:686] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [term 1 LEADER]: Becoming Leader. State: Replica: 14df8fa233694ec5ad1e233001c9631e, State: Running, Role: LEADER
I20260806 06:39:43.595088 22598 consensus_queue.cc:237] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8418d90ccc7142f4aa467a9b86a9f860" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38843 } } peers { permanent_uuid: "14df8fa233694ec5ad1e233001c9631e" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42961 } } peers { permanent_uuid: "5e5b96e52b4142d5b6caee27a681b79d" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 33217 } }
I20260806 06:39:43.596029 21929 catalog_manager.cc:5495] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e reported cstate change: term changed from 0 to 1, leader changed from <none> to 14df8fa233694ec5ad1e233001c9631e (127.29.142.129). New cstate: current_term: 1 leader_uuid: "14df8fa233694ec5ad1e233001c9631e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8418d90ccc7142f4aa467a9b86a9f860" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14df8fa233694ec5ad1e233001c9631e" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 42961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5b96e52b4142d5b6caee27a681b79d" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 33217 } health_report { overall_health: UNKNOWN } } }
I20260806 06:39:43.608323 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.605640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46124 (local address 127.29.142.129:42961)
0806 06:39:43.605685 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.605688 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.605697 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:39:43.605803 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.605804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.605826 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.605923 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:39:43.606077 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.606570 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.607436 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.607674 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.607769 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.607957 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:39:43.608067 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:39:43.608162 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.608182 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.608203 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:39:43.610376 22305 raft_consensus.cc:1261] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860 [term 1 FOLLOWER]: Refusing update from remote peer 14df8fa233694ec5ad1e233001c9631e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:43.610378 22493 raft_consensus.cc:1261] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d [term 1 FOLLOWER]: Refusing update from remote peer 14df8fa233694ec5ad1e233001c9631e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:43.610798 22598 consensus_queue.cc:1035] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8418d90ccc7142f4aa467a9b86a9f860" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:43.610966 22604 consensus_queue.cc:1035] T c09ac8885f7d42cb9d837d3d0d5352b5 P 14df8fa233694ec5ad1e233001c9631e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5b96e52b4142d5b6caee27a681b79d" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 33217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:43.613936 22612 mvcc.cc:205] Tried to move back new op lower bound from 7315449379264872448 to 7315449379206246400. Current Snapshot: MvccSnapshot[applied={T|T < 7315449379264872448}]
I20260806 06:39:43.614789 22609 mvcc.cc:205] Tried to move back new op lower bound from 7315449379264872448 to 7315449379206246400. Current Snapshot: MvccSnapshot[applied={T|T < 7315449379264872448 or (T in {7315449379264872448})}]
I20260806 06:39:43.616360 21930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:43.616473 21930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:43.617292 21930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 102a26ce95314edf865422103bdeb4f4: Sending DeleteTablet for 3 replicas of tablet c09ac8885f7d42cb9d837d3d0d5352b5
I20260806 06:39:43.617661 22097 tablet_service.cc:1514] Processing DeleteTablet for tablet c09ac8885f7d42cb9d837d3d0d5352b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:43 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:46108
I20260806 06:39:43.617676 22464 tablet_service.cc:1514] Processing DeleteTablet for tablet c09ac8885f7d42cb9d837d3d0d5352b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:43 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:44826
I20260806 06:39:43.617920 22624 tablet_replica.cc:330] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d: stopping tablet replica
I20260806 06:39:43.617939 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 22009
I20260806 06:39:43.618028 22624 raft_consensus.cc:2229] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:39:43.618136 22624 raft_consensus.cc:2258] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:43.618446 22279 tablet_service.cc:1514] Processing DeleteTablet for tablet c09ac8885f7d42cb9d837d3d0d5352b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:43 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:46894
I20260806 06:39:43.618746 22627 tablet_replica.cc:330] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860: stopping tablet replica
I20260806 06:39:43.618886 22627 raft_consensus.cc:2229] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:39:43.619012 22627 raft_consensus.cc:2258] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:43.622509 22627 ts_tablet_manager.cc:1905] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:43.622941 22624 ts_tablet_manager.cc:1905] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:43.624600 22627 ts_tablet_manager.cc:1918] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:43.624696 22627 log.cc:1196] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/c09ac8885f7d42cb9d837d3d0d5352b5
I20260806 06:39:43.624995 22624 ts_tablet_manager.cc:1918] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:43.625013 22627 ts_tablet_manager.cc:1939] T c09ac8885f7d42cb9d837d3d0d5352b5 P 8418d90ccc7142f4aa467a9b86a9f860: Deleting consensus metadata
I20260806 06:39:43.625104 22624 log.cc:1196] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/c09ac8885f7d42cb9d837d3d0d5352b5
I20260806 06:39:43.625377 22624 ts_tablet_manager.cc:1939] T c09ac8885f7d42cb9d837d3d0d5352b5 P 5e5b96e52b4142d5b6caee27a681b79d: Deleting consensus metadata
I20260806 06:39:43.625569 21909 catalog_manager.cc:4841] TS 8418d90ccc7142f4aa467a9b86a9f860 (127.29.142.130:38843): tablet c09ac8885f7d42cb9d837d3d0d5352b5 (table test-table [id=12eae8c63ab14d969d8b8b03331858e9]) successfully deleted
I20260806 06:39:43.625833 21909 catalog_manager.cc:4841] TS 5e5b96e52b4142d5b6caee27a681b79d (127.29.142.131:33217): tablet c09ac8885f7d42cb9d837d3d0d5352b5 (table test-table [id=12eae8c63ab14d969d8b8b03331858e9]) successfully deleted
W20260806 06:39:43.630923 21913 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:42961 (error 108)
I20260806 06:39:43.631222 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 22195
I20260806 06:39:43.636457 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.636218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:42961 (local address 127.0.0.1:46138)
0806 06:39:43.636323 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:43.636394 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:42961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260806 06:39:43.636734 21913 catalog_manager.cc:4584] TS 14df8fa233694ec5ad1e233001c9631e (127.29.142.129:42961): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c09ac8885f7d42cb9d837d3d0d5352b5: Network error: Client connection negotiation failed: client connection to 127.29.142.129:42961: connect: Connection refused (error 111)
I20260806 06:39:43.644827 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 22380
I20260806 06:39:43.657346 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 21888
2026-08-06T06:39:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2127 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T06:39:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:39:43Z Disabled control of system clock
I20260806 06:39:43.693413 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:38509
--webserver_interface=127.29.142.190
--webserver_port=0
--builtin_ntp_servers=127.29.142.148:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:38509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:39:43.780272 22633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:43.780526 22633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:43.780550 22633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:43.781687 22633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:43.781725 22633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:43.781738 22633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:43.781751 22633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:43.781764 22633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:43.783208 22633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:38509
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:38509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:43.783438 22633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:43.783718 22633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:43.788975 22640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:43.788971 22642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:43.788975 22639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:43.789247 22633 server_base.cc:1034] running on GCE node
I20260806 06:39:43.789486 22633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:43.789764 22633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:43.790949 22633 hybrid_clock.cc:648] HybridClock initialized: now 1785998383790926 us; error 45 us; skew 500 ppm
I20260806 06:39:43.792517 22633 webserver.cc:460] Webserver started at http://127.29.142.190:33119/ using document root <none> and password file <none>
I20260806 06:39:43.792929 22633 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:43.793006 22633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:43.793154 22633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:43.794549 22633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "91b30f41fdd847608ca13fe2aec3e2a8"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "8d221ba1b5bb0413690f94a512ea571d"
server_key_iv: "98f1493038cc8240cb0613ef778e2577"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.795054 22633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "91b30f41fdd847608ca13fe2aec3e2a8"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "8d221ba1b5bb0413690f94a512ea571d"
server_key_iv: "98f1493038cc8240cb0613ef778e2577"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.796496 22633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:43.797374 22650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.797600 22633 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:43.797713 22633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "91b30f41fdd847608ca13fe2aec3e2a8"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "8d221ba1b5bb0413690f94a512ea571d"
server_key_iv: "98f1493038cc8240cb0613ef778e2577"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.797832 22633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:43.808833 22633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:43.809218 22633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:43.809394 22633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:43.813774 22633 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:38509
I20260806 06:39:43.813815 22727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:38509 every 8 connection(s)
I20260806 06:39:43.814194 22633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:43.815482 22728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:43.818012 22728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8: Bootstrap starting.
I20260806 06:39:43.818466 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 22633
I20260806 06:39:43.818584 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:39:43.818871 30266 external_mini_cluster.cc:1428] Setting key a708318b9f912e394325be8f38c07d37
I20260806 06:39:43.818869 22728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:43.819262 22728 log.cc:824] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:43.820173 22728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8: No bootstrap required, opened a new log
I20260806 06:39:43.822428 22728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b30f41fdd847608ca13fe2aec3e2a8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38509 } }
I20260806 06:39:43.822717 22728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:43.822767 22728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b30f41fdd847608ca13fe2aec3e2a8, State: Initialized, Role: FOLLOWER
I20260806 06:39:43.822973 22728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b30f41fdd847608ca13fe2aec3e2a8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38509 } }
I20260806 06:39:43.823087 22728 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:43.823122 22728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:43.823170 22728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:43.823966 22728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b30f41fdd847608ca13fe2aec3e2a8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38509 } }
I20260806 06:39:43.824115 22728 leader_election.cc:302] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91b30f41fdd847608ca13fe2aec3e2a8; no voters:
I20260806 06:39:43.824362 22728 leader_election.cc:288] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:39:43.824397 22736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:43.824577 22736 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [term 1 LEADER]: Becoming Leader. State: Replica: 91b30f41fdd847608ca13fe2aec3e2a8, State: Running, Role: LEADER
I20260806 06:39:43.824678 22736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b30f41fdd847608ca13fe2aec3e2a8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38509 } }
I20260806 06:39:43.824723 22728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:43.825057 22737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91b30f41fdd847608ca13fe2aec3e2a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b30f41fdd847608ca13fe2aec3e2a8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38509 } } }
I20260806 06:39:43.825165 22737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:43.825464 22738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91b30f41fdd847608ca13fe2aec3e2a8. Latest consensus state: current_term: 1 leader_uuid: "91b30f41fdd847608ca13fe2aec3e2a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b30f41fdd847608ca13fe2aec3e2a8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 38509 } } }
I20260806 06:39:43.825600 22738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:43.825776 22741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:43.826568 22741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:39:43.827157 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.819482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39232 (local address 127.29.142.190:38509)
0806 06:39:43.819719 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.819727 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.819755 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:39:43.820440 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.820450 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.820537 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.820733 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:43.820996 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.823420 (+ 2424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.824403 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.825060 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.825592 (+ 532us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.826078 (+ 486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:43.826082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:43.826082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:43.826318 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:43.826363 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:43.826378 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:43.826478 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.826852 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.826930 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260806 06:39:43.828418 22741 catalog_manager.cc:1347] Generated new cluster ID: 61d17a1b100f436f9b4bb09dcf3b1da4
I20260806 06:39:43.828485 22741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:39:43.833108 22741 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:39:43.833884 22741 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:43.843233 22741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8: Generated new TSK 0
I20260806 06:39:43.843472 22741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:39:43.846256 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:38509
--builtin_ntp_servers=127.29.142.148:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:39:43.931866 22759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:43.932121 22759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:43.932163 22759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:43.933347 22759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:43.933416 22759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:43.933461 22759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:39:43.935081 22759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:38509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:43.935359 22759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:43.935683 22759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:43.941159 22767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:43.941162 22771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:43.941162 22768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:43.941526 22759 server_base.cc:1034] running on GCE node
I20260806 06:39:43.941779 22759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:43.942060 22759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:43.943249 22759 hybrid_clock.cc:648] HybridClock initialized: now 1785998383943221 us; error 47 us; skew 500 ppm
I20260806 06:39:43.944911 22759 webserver.cc:460] Webserver started at http://127.29.142.129:36255/ using document root <none> and password file <none>
I20260806 06:39:43.945214 22759 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:43.945298 22759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:43.945474 22759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:43.946833 22759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "0905b6f5f6c045d8afd84b56cb61dff5"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "8b2be9e384a635b543e3ca5605e8ccc2"
server_key_iv: "fc61445d2821b609eedacb48a1925e9f"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.947283 22759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "0905b6f5f6c045d8afd84b56cb61dff5"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "8b2be9e384a635b543e3ca5605e8ccc2"
server_key_iv: "fc61445d2821b609eedacb48a1925e9f"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.949079 22759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:39:43.950071 22778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.950296 22759 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:43.950425 22759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "0905b6f5f6c045d8afd84b56cb61dff5"
format_stamp: "Formatted at 2026-08-06 06:39:43 on dist-test-slave-3kk6"
server_key: "8b2be9e384a635b543e3ca5605e8ccc2"
server_key_iv: "fc61445d2821b609eedacb48a1925e9f"
server_key_version: "encryptionkey@0"
I20260806 06:39:43.950517 22759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:43.972415 22759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:43.972815 22759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:43.973001 22759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:43.973287 22759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:43.973694 22759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:43.973760 22759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.973812 22759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:43.973840 22759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:43.980449 22759 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:36553
I20260806 06:39:43.980501 22932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:36553 every 8 connection(s)
I20260806 06:39:43.980914 22759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:39:43.982714 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 22759
I20260806 06:39:43.982844 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:39:43.983064 30266 external_mini_cluster.cc:1428] Setting key a101c3c9ae8c1f9f69c9e07c2fc2e6e8
I20260806 06:39:43.984109 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:38509
--builtin_ntp_servers=127.29.142.148:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:39:43.987797 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.981889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:49663 (local address 127.29.142.190:38509)
0806 06:39:43.981965 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:43.981967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:43.982167 (+ 200us) server_negotiation.cc:407] Connection header received
0806 06:39:43.982852 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:43.982854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:43.982867 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:43.982953 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:43.983945 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.984530 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.986085 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.986420 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.986504 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.987528 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:43.987530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:43.987530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:43.987535 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:43.987548 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:43.987552 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:43.987606 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:43.987707 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 06:39:43.987730 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:39:43.988111 22936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:43.981800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:38509 (local address 127.29.142.129:49663)
0806 06:39:43.982029 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:43.982076 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:43.982720 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:43.983050 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:43.983055 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:43.983065 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:43.983853 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.983862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.984638 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.984639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.985993 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:43.985996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:43.986519 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:43.986520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:43.986635 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:43.987362 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:39:43.987363 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:43.987405 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:39:43.987437 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:43.987624 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:43.987631 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:43.987634 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:39:43.987706 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:39:43.987833 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
I20260806 06:39:43.988705 22933 heartbeater.cc:347] Connected to a master server at 127.29.142.190:38509
I20260806 06:39:43.988833 22933 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:43.989063 22933 heartbeater.cc:511] Master 127.29.142.190:38509 requested a full tablet report, sending...
I20260806 06:39:43.989614 22673 ts_manager.cc:194] Registered new tserver with Master: 0905b6f5f6c045d8afd84b56cb61dff5 (127.29.142.129:36553)
I20260806 06:39:43.990242 22673 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.142.129:49663
W20260806 06:39:44.070842 22937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:44.071118 22937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:44.071142 22937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:44.072410 22937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:44.072466 22937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:44.072494 22937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:39:44.073916 22937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:38509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:44.074213 22937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:44.074558 22937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:44.079942 22948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:44.080060 22937 server_base.cc:1034] running on GCE node
W20260806 06:39:44.079934 22947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:44.080121 22950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:44.080363 22937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:44.080686 22937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:44.081858 22937 hybrid_clock.cc:648] HybridClock initialized: now 1785998384081834 us; error 37 us; skew 500 ppm
I20260806 06:39:44.083261 22937 webserver.cc:460] Webserver started at http://127.29.142.130:42443/ using document root <none> and password file <none>
I20260806 06:39:44.083523 22937 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:44.083598 22937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:44.083770 22937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:44.084969 22937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "affa3a2f74674cc4af30b0de1d6b6fbc"
format_stamp: "Formatted at 2026-08-06 06:39:44 on dist-test-slave-3kk6"
server_key: "f687270e7c6378c660ee991a29ad07a9"
server_key_iv: "706513456ab42dfe4300d298ba996b6e"
server_key_version: "encryptionkey@0"
I20260806 06:39:44.085367 22937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "affa3a2f74674cc4af30b0de1d6b6fbc"
format_stamp: "Formatted at 2026-08-06 06:39:44 on dist-test-slave-3kk6"
server_key: "f687270e7c6378c660ee991a29ad07a9"
server_key_iv: "706513456ab42dfe4300d298ba996b6e"
server_key_version: "encryptionkey@0"
I20260806 06:39:44.086794 22937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260806 06:39:44.087604 22958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:44.087788 22937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:44.087903 22937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "affa3a2f74674cc4af30b0de1d6b6fbc"
format_stamp: "Formatted at 2026-08-06 06:39:44 on dist-test-slave-3kk6"
server_key: "f687270e7c6378c660ee991a29ad07a9"
server_key_iv: "706513456ab42dfe4300d298ba996b6e"
server_key_version: "encryptionkey@0"
I20260806 06:39:44.087986 22937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:44.104422 22937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:44.104810 22937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:44.104990 22937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:44.105291 22937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:44.105666 22937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:44.105726 22937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:44.105772 22937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:44.105800 22937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:44.111910 22937 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:38061
I20260806 06:39:44.112015 23125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:38061 every 8 connection(s)
I20260806 06:39:44.112393 22937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:39:44.117960 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.113255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:50285 (local address 127.29.142.190:38509)
0806 06:39:44.113309 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:44.113312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:44.113580 (+ 268us) server_negotiation.cc:407] Connection header received
0806 06:39:44.114090 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:44.114091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:44.114101 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:44.114159 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:44.114933 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.115445 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.116460 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.116652 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.116720 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.117675 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:44.117677 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:44.117677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:44.117682 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:44.117698 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:44.117702 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:44.117759 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:44.117864 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:39:44.117888 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:39:44.118144 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.113254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:38509 (local address 127.29.142.130:50285)
0806 06:39:44.113448 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:44.113486 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:44.114000 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:44.114214 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:44.114218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:44.114226 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:44.114848 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.114854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.115533 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.115534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.116354 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.116356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.116742 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.116742 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.116869 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.117502 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:39:44.117503 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:44.117534 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:39:44.117559 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:44.117787 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:44.117792 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:44.117795 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:39:44.117851 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:39:44.117936 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260806 06:39:44.118705 23126 heartbeater.cc:347] Connected to a master server at 127.29.142.190:38509
I20260806 06:39:44.118848 23126 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:44.119047 23126 heartbeater.cc:511] Master 127.29.142.190:38509 requested a full tablet report, sending...
I20260806 06:39:44.119467 22673 ts_manager.cc:194] Registered new tserver with Master: affa3a2f74674cc4af30b0de1d6b6fbc (127.29.142.130:38061)
I20260806 06:39:44.119874 22673 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.142.130:50285
I20260806 06:39:44.121382 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 22937
I20260806 06:39:44.121510 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:39:44.121704 30266 external_mini_cluster.cc:1428] Setting key dcad0d24564952ec4ac4b33003872d83
I20260806 06:39:44.122692 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:38509
--builtin_ntp_servers=127.29.142.148:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:39:44.207933 23131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:44.208164 23131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:44.208189 23131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:44.209336 23131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:44.209384 23131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:44.209411 23131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:39:44.210862 23131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:38509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:44.211078 23131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:44.211365 23131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:44.216694 23142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:44.216694 23144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:44.216719 23141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:44.216866 23131 server_base.cc:1034] running on GCE node
I20260806 06:39:44.217212 23131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:44.217495 23131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:44.218706 23131 hybrid_clock.cc:648] HybridClock initialized: now 1785998384218678 us; error 42 us; skew 500 ppm
I20260806 06:39:44.220247 23131 webserver.cc:460] Webserver started at http://127.29.142.131:36799/ using document root <none> and password file <none>
I20260806 06:39:44.220541 23131 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:44.220621 23131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:44.220801 23131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:44.221957 23131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "a8429d24b74949029e13bc4e6c569ac1"
format_stamp: "Formatted at 2026-08-06 06:39:44 on dist-test-slave-3kk6"
server_key: "0d531a67b3f210ba777ee1b2dc58c217"
server_key_iv: "f80ea88e67ebf997d8a4321ac2258c66"
server_key_version: "encryptionkey@0"
I20260806 06:39:44.222357 23131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "a8429d24b74949029e13bc4e6c569ac1"
format_stamp: "Formatted at 2026-08-06 06:39:44 on dist-test-slave-3kk6"
server_key: "0d531a67b3f210ba777ee1b2dc58c217"
server_key_iv: "f80ea88e67ebf997d8a4321ac2258c66"
server_key_version: "encryptionkey@0"
I20260806 06:39:44.224157 23131 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 06:39:44.225051 23155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:44.225282 23131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:44.225380 23131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "a8429d24b74949029e13bc4e6c569ac1"
format_stamp: "Formatted at 2026-08-06 06:39:44 on dist-test-slave-3kk6"
server_key: "0d531a67b3f210ba777ee1b2dc58c217"
server_key_iv: "f80ea88e67ebf997d8a4321ac2258c66"
server_key_version: "encryptionkey@0"
I20260806 06:39:44.225466 23131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:44.236989 23131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:44.237349 23131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:44.237512 23131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:44.237792 23131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:44.238150 23131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:44.238191 23131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:44.238215 23131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:44.238229 23131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:44.244042 23131 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:36657
I20260806 06:39:44.244118 23310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:36657 every 8 connection(s)
I20260806 06:39:44.244491 23131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:39:44.248574 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 23131
I20260806 06:39:44.248706 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:39:44.248932 30266 external_mini_cluster.cc:1428] Setting key 2779304d99d83a905d54cb98f672e83d
I20260806 06:39:44.250319 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.245382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:39857 (local address 127.29.142.190:38509)
0806 06:39:44.245451 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:44.245453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:44.245658 (+ 205us) server_negotiation.cc:407] Connection header received
0806 06:39:44.246194 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:44.246195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:44.246205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:44.246297 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:44.247036 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.247534 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.248633 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.248924 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.249003 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.250108 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:44.250109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:44.250110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:44.250114 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:44.250128 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:44.250132 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:44.250176 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:44.250225 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:39:44.250258 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:39:44.250530 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.245297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:38509 (local address 127.29.142.131:39857)
0806 06:39:44.245517 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:44.245561 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:44.246100 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:44.246393 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:44.246396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:44.246405 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:44.246949 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.246953 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.247616 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.247616 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.248535 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.248538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.249037 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.249039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.249223 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.249958 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:39:44.249958 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:44.249998 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:39:44.250032 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:44.250177 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:44.250179 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:44.250181 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:39:44.250229 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:39:44.250314 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260806 06:39:44.250998 23311 heartbeater.cc:347] Connected to a master server at 127.29.142.190:38509
I20260806 06:39:44.251094 23311 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:44.251246 23311 heartbeater.cc:511] Master 127.29.142.190:38509 requested a full tablet report, sending...
I20260806 06:39:44.251619 22673 ts_manager.cc:194] Registered new tserver with Master: a8429d24b74949029e13bc4e6c569ac1 (127.29.142.131:36657)
I20260806 06:39:44.252007 22673 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.142.131:39857
I20260806 06:39:44.261037 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:39:44.265529 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.262604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39242 (local address 127.29.142.190:38509)
0806 06:39:44.262693 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:44.262695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:44.262804 (+ 109us) server_negotiation.cc:407] Connection header received
0806 06:39:44.262858 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:44.262859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:44.262869 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:44.262930 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:44.263069 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.263549 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.264323 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.264510 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.264570 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.265295 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:44.265297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:44.265297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:44.265302 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:44.265315 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:44.265319 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:44.265365 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:44.265440 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 06:39:44.265464 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:39:44.267484 22673 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:39:44.269050 22673 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:39:44.276728 23330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.271711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:36657 (local address 127.0.0.1:35364)
0806 06:39:44.272807 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:44.272818 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:44.272886 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:44.273161 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:44.273162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:44.273164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:44.273219 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.273221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.274525 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.274526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.275368 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.275371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.276355 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.276356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.276447 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.276485 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 06:39:44.276556 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:39:44.276616 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1083,"thread_start_us":171,"threads_started":1}
I20260806 06:39:44.276914 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.271999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.29.142.130:38061)
0806 06:39:44.272207 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:44.272212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:44.272931 (+ 719us) server_negotiation.cc:407] Connection header received
0806 06:39:44.273124 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:44.273126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:44.273144 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:44.273280 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:44.273493 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.274628 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.275842 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.276391 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.276465 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.276557 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:44.276771 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 06:39:44.276826 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":94,"threads_started":1}
I20260806 06:39:44.276997 23326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.271693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:36553 (local address 127.0.0.1:56874)
0806 06:39:44.271986 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:44.272064 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:44.272162 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:44.272866 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:44.272870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:44.272872 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:44.272954 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.272956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.274201 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.274202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.275522 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.275525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.276727 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.276728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.276811 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.276818 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:39:44.276855 (+ 37us) client_negotiation.cc:240] Negotiation successful
0806 06:39:44.276908 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":4,"thread_start_us":92,"threads_started":1}
I20260806 06:39:44.277029 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.272186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35364 (local address 127.29.142.131:36657)
0806 06:39:44.272460 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:44.272464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:44.272951 (+ 487us) server_negotiation.cc:407] Connection header received
0806 06:39:44.273000 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:44.273003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:44.273021 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:44.273094 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:44.273300 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.274416 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.275474 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.276238 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.276339 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.276444 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:44.276886 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 06:39:44.276935 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":99,"threads_started":1}
I20260806 06:39:44.277113 23336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.271697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:38061 (local address 127.0.0.1:56924)
0806 06:39:44.272791 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:44.272833 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:44.272886 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:44.273377 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:44.273378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:44.273380 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:44.273413 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.273414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.274724 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.274725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.275744 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.275746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.276595 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.276596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.276646 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.276648 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:39:44.276696 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:39:44.276727 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1070,"thread_start_us":112,"threads_started":1}
I20260806 06:39:44.277498 23335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.271887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56874 (local address 127.29.142.129:36553)
0806 06:39:44.272653 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:44.272658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:44.272669 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:39:44.272712 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:44.272713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:44.272730 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:44.273032 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:44.273095 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.274109 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.275621 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.276627 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.277215 (+ 588us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.277337 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:44.277369 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 06:39:44.277418 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":686,"threads_started":1}
I20260806 06:39:44.277786 23031 tablet_service.cc:1467] Processing CreateTablet for tablet 211fddfcfc5142449991a4149d7656a7 (DEFAULT_TABLE table=test-table [id=1a22b202afd44b76a8fe70987fabf58a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:44.277786 23220 tablet_service.cc:1467] Processing CreateTablet for tablet 211fddfcfc5142449991a4149d7656a7 (DEFAULT_TABLE table=test-table [id=1a22b202afd44b76a8fe70987fabf58a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:44.278200 23031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 211fddfcfc5142449991a4149d7656a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:44.278200 23220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 211fddfcfc5142449991a4149d7656a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:44.278129 22840 tablet_service.cc:1467] Processing CreateTablet for tablet 211fddfcfc5142449991a4149d7656a7 (DEFAULT_TABLE table=test-table [id=1a22b202afd44b76a8fe70987fabf58a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:44.278503 22840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 211fddfcfc5142449991a4149d7656a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:44.281317 23337 tablet_bootstrap.cc:492] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5: Bootstrap starting.
I20260806 06:39:44.281955 23339 tablet_bootstrap.cc:492] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1: Bootstrap starting.
I20260806 06:39:44.282018 23338 tablet_bootstrap.cc:492] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc: Bootstrap starting.
I20260806 06:39:44.282168 23337 tablet_bootstrap.cc:654] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:44.282495 23337 log.cc:824] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:44.282677 23338 tablet_bootstrap.cc:654] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:44.282677 23339 tablet_bootstrap.cc:654] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:44.283030 23338 log.cc:824] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:44.283030 23339 log.cc:824] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:44.283413 23337 tablet_bootstrap.cc:492] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5: No bootstrap required, opened a new log
I20260806 06:39:44.283504 23337 ts_tablet_manager.cc:1397] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:39:44.284009 23338 tablet_bootstrap.cc:492] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc: No bootstrap required, opened a new log
I20260806 06:39:44.284106 23338 ts_tablet_manager.cc:1397] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:39:44.284129 23339 tablet_bootstrap.cc:492] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1: No bootstrap required, opened a new log
I20260806 06:39:44.284214 23339 ts_tablet_manager.cc:1397] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:39:44.286182 23338 raft_consensus.cc:348] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0905b6f5f6c045d8afd84b56cb61dff5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 36553 } } peers { permanent_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38061 } } peers { permanent_uuid: "a8429d24b74949029e13bc4e6c569ac1" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 36657 } }
I20260806 06:39:44.286374 23338 raft_consensus.cc:374] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:44.286427 23338 raft_consensus.cc:729] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: affa3a2f74674cc4af30b0de1d6b6fbc, State: Initialized, Role: FOLLOWER
I20260806 06:39:44.286347 23337 raft_consensus.cc:348] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0905b6f5f6c045d8afd84b56cb61dff5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 36553 } } peers { permanent_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38061 } } peers { permanent_uuid: "a8429d24b74949029e13bc4e6c569ac1" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 36657 } }
I20260806 06:39:44.286530 23337 raft_consensus.cc:374] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:44.286576 23337 raft_consensus.cc:729] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0905b6f5f6c045d8afd84b56cb61dff5, State: Initialized, Role: FOLLOWER
I20260806 06:39:44.286545 23338 consensus_queue.cc:260] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0905b6f5f6c045d8afd84b56cb61dff5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 36553 } } peers { permanent_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38061 } } peers { permanent_uuid: "a8429d24b74949029e13bc4e6c569ac1" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 36657 } }
I20260806 06:39:44.286753 23337 consensus_queue.cc:260] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0905b6f5f6c045d8afd84b56cb61dff5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 36553 } } peers { permanent_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38061 } } peers { permanent_uuid: "a8429d24b74949029e13bc4e6c569ac1" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 36657 } }
I20260806 06:39:44.286878 23338 ts_tablet_manager.cc:1428] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 06:39:44.286948 23126 heartbeater.cc:503] Master 127.29.142.190:38509 was elected leader, sending a full tablet report...
I20260806 06:39:44.287057 23337 ts_tablet_manager.cc:1428] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260806 06:39:44.286988 23339 raft_consensus.cc:348] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0905b6f5f6c045d8afd84b56cb61dff5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 36553 } } peers { permanent_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38061 } } peers { permanent_uuid: "a8429d24b74949029e13bc4e6c569ac1" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 36657 } }
I20260806 06:39:44.287132 23339 raft_consensus.cc:374] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:44.287171 23339 raft_consensus.cc:729] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8429d24b74949029e13bc4e6c569ac1, State: Initialized, Role: FOLLOWER
I20260806 06:39:44.287220 22933 heartbeater.cc:503] Master 127.29.142.190:38509 was elected leader, sending a full tablet report...
I20260806 06:39:44.287287 23339 consensus_queue.cc:260] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0905b6f5f6c045d8afd84b56cb61dff5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 36553 } } peers { permanent_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38061 } } peers { permanent_uuid: "a8429d24b74949029e13bc4e6c569ac1" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 36657 } }
I20260806 06:39:44.287578 23339 ts_tablet_manager.cc:1428] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:39:44.287619 23311 heartbeater.cc:503] Master 127.29.142.190:38509 was elected leader, sending a full tablet report...
W20260806 06:39:44.363041 23127 tablet.cc:2369] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:39:44.481741 22935 tablet.cc:2369] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:39:44.495162 23316 tablet.cc:2369] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:39:44.534557 23347 raft_consensus.cc:482] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:39:44.534782 23347 raft_consensus.cc:504] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0905b6f5f6c045d8afd84b56cb61dff5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 36553 } } peers { permanent_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38061 } } peers { permanent_uuid: "a8429d24b74949029e13bc4e6c569ac1" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 36657 } }
I20260806 06:39:44.535149 23347 leader_election.cc:288] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0905b6f5f6c045d8afd84b56cb61dff5 (127.29.142.129:36553), a8429d24b74949029e13bc4e6c569ac1 (127.29.142.131:36657)
I20260806 06:39:44.538126 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.535392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:36553 (local address 127.29.142.130:33311)
0806 06:39:44.535447 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:44.535464 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:44.535568 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:44.535764 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:44.535765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:44.535767 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:44.535856 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.535858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.536521 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.536522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.537284 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.537287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.537851 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.537852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.537933 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.537936 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:39:44.537986 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:39:44.538031 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:39:44.538142 23335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.535534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:33311 (local address 127.29.142.129:36553)
0806 06:39:44.535598 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:44.535601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:44.535629 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:39:44.535664 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:44.535665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:44.535686 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:44.535733 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:44.535959 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.536426 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.537388 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.537756 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.537852 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.537861 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:44.538018 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 06:39:44.538068 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:39:44.538426 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.535484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:53301 (local address 127.29.142.131:36657)
0806 06:39:44.535538 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:44.535540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:44.535806 (+ 266us) server_negotiation.cc:407] Connection header received
0806 06:39:44.535866 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:44.535867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:44.535874 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:44.535942 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:44.536126 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.536589 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.537532 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.538003 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.538073 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.538079 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:44.538311 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 06:39:44.538363 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:39:44.538558 22868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211fddfcfc5142449991a4149d7656a7" candidate_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0905b6f5f6c045d8afd84b56cb61dff5" is_pre_election: true
I20260806 06:39:44.538688 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.535392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:36657 (local address 127.29.142.130:53301)
0806 06:39:44.535710 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:44.535740 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:44.535808 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:44.535964 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:44.535965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:44.535968 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:44.536048 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.536050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.536708 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.536709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.537432 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:44.537434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.538122 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.538123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:44.538212 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.538216 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:39:44.538532 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 06:39:44.538572 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":75,"thread_start_us":158,"threads_started":1}
I20260806 06:39:44.538795 22868 raft_consensus.cc:2454] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate affa3a2f74674cc4af30b0de1d6b6fbc in term 0.
I20260806 06:39:44.539103 22963 leader_election.cc:302] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0905b6f5f6c045d8afd84b56cb61dff5, affa3a2f74674cc4af30b0de1d6b6fbc; no voters:
I20260806 06:39:44.539251 23249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211fddfcfc5142449991a4149d7656a7" candidate_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8429d24b74949029e13bc4e6c569ac1" is_pre_election: true
I20260806 06:39:44.539263 23347 raft_consensus.cc:2781] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:39:44.539453 23249 raft_consensus.cc:2454] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate affa3a2f74674cc4af30b0de1d6b6fbc in term 0.
I20260806 06:39:44.539481 23347 raft_consensus.cc:482] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:39:44.539537 23347 raft_consensus.cc:3037] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:44.540357 23347 raft_consensus.cc:504] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0905b6f5f6c045d8afd84b56cb61dff5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 36553 } } peers { permanent_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38061 } } peers { permanent_uuid: "a8429d24b74949029e13bc4e6c569ac1" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 36657 } }
I20260806 06:39:44.540514 23347 leader_election.cc:288] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [CANDIDATE]: Term 1 election: Requested vote from peers 0905b6f5f6c045d8afd84b56cb61dff5 (127.29.142.129:36553), a8429d24b74949029e13bc4e6c569ac1 (127.29.142.131:36657)
I20260806 06:39:44.540685 23249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211fddfcfc5142449991a4149d7656a7" candidate_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8429d24b74949029e13bc4e6c569ac1"
I20260806 06:39:44.540678 22868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "211fddfcfc5142449991a4149d7656a7" candidate_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0905b6f5f6c045d8afd84b56cb61dff5"
I20260806 06:39:44.540768 23249 raft_consensus.cc:3037] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:44.540786 22868 raft_consensus.cc:3037] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:44.541551 22868 raft_consensus.cc:2454] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate affa3a2f74674cc4af30b0de1d6b6fbc in term 1.
I20260806 06:39:44.541581 23249 raft_consensus.cc:2454] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate affa3a2f74674cc4af30b0de1d6b6fbc in term 1.
I20260806 06:39:44.541728 22963 leader_election.cc:302] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0905b6f5f6c045d8afd84b56cb61dff5, affa3a2f74674cc4af30b0de1d6b6fbc; no voters:
I20260806 06:39:44.541841 23347 raft_consensus.cc:2781] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:44.541994 23347 raft_consensus.cc:686] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 1 LEADER]: Becoming Leader. State: Replica: affa3a2f74674cc4af30b0de1d6b6fbc, State: Running, Role: LEADER
I20260806 06:39:44.542093 23347 consensus_queue.cc:237] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0905b6f5f6c045d8afd84b56cb61dff5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 36553 } } peers { permanent_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38061 } } peers { permanent_uuid: "a8429d24b74949029e13bc4e6c569ac1" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 36657 } }
I20260806 06:39:44.542910 22671 catalog_manager.cc:5495] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc reported cstate change: term changed from 0 to 1, leader changed from <none> to affa3a2f74674cc4af30b0de1d6b6fbc (127.29.142.130). New cstate: current_term: 1 leader_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0905b6f5f6c045d8afd84b56cb61dff5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 36553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "affa3a2f74674cc4af30b0de1d6b6fbc" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8429d24b74949029e13bc4e6c569ac1" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 36657 } health_report { overall_health: UNKNOWN } } }
I20260806 06:39:44.562845 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.560343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56930 (local address 127.29.142.130:38061)
0806 06:39:44.560406 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:44.560408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:44.560417 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:39:44.560457 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:44.560458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:44.560476 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:44.560539 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:39:44.560699 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.561159 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.561928 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:44.562112 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:44.562176 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:44.562374 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:39:44.562522 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:39:44.562593 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:44.562711 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 06:39:44.562735 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:39:44.565029 22868 raft_consensus.cc:1261] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5 [term 1 FOLLOWER]: Refusing update from remote peer affa3a2f74674cc4af30b0de1d6b6fbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:44.565029 23249 raft_consensus.cc:1261] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1 [term 1 FOLLOWER]: Refusing update from remote peer affa3a2f74674cc4af30b0de1d6b6fbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:44.565475 23351 consensus_queue.cc:1035] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8429d24b74949029e13bc4e6c569ac1" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 36657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:44.565629 23347 consensus_queue.cc:1035] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0905b6f5f6c045d8afd84b56cb61dff5" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 36553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:44.568825 23356 mvcc.cc:205] Tried to move back new op lower bound from 7315449383174987776 to 7315449383084867584. Current Snapshot: MvccSnapshot[applied={T|T < 7315449383174987776}]
I20260806 06:39:44.570796 22671 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:44.570910 22671 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:44.571746 22671 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91b30f41fdd847608ca13fe2aec3e2a8: Sending DeleteTablet for 3 replicas of tablet 211fddfcfc5142449991a4149d7656a7
I20260806 06:39:44.572131 22840 tablet_service.cc:1514] Processing DeleteTablet for tablet 211fddfcfc5142449991a4149d7656a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:44 UTC) from {username='slave'} at 127.0.0.1:56874
I20260806 06:39:44.572167 23031 tablet_service.cc:1514] Processing DeleteTablet for tablet 211fddfcfc5142449991a4149d7656a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:44 UTC) from {username='slave'} at 127.0.0.1:56924
I20260806 06:39:44.572330 23220 tablet_service.cc:1514] Processing DeleteTablet for tablet 211fddfcfc5142449991a4149d7656a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:44 UTC) from {username='slave'} at 127.0.0.1:35364
I20260806 06:39:44.572605 23370 tablet_replica.cc:330] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1: stopping tablet replica
I20260806 06:39:44.572664 23366 tablet_replica.cc:330] T 211fddfcfc5142449991a4149d7656a7 P 0905b6f5f6c045d8afd84b56cb61dff5: stopping tablet replica
I20260806 06:39:44.572836 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 22759
I20260806 06:39:44.578878 23370 raft_consensus.cc:2229] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:39:44.579130 23370 raft_consensus.cc:2258] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:44.580353 23370 ts_tablet_manager.cc:1905] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:44.582377 23370 ts_tablet_manager.cc:1918] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:44.582507 23370 log.cc:1196] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/211fddfcfc5142449991a4149d7656a7
I20260806 06:39:44.582831 23370 ts_tablet_manager.cc:1939] T 211fddfcfc5142449991a4149d7656a7 P a8429d24b74949029e13bc4e6c569ac1: Deleting consensus metadata
I20260806 06:39:44.584003 22652 catalog_manager.cc:4841] TS a8429d24b74949029e13bc4e6c569ac1 (127.29.142.131:36657): tablet 211fddfcfc5142449991a4149d7656a7 (table test-table [id=1a22b202afd44b76a8fe70987fabf58a]) successfully deleted
I20260806 06:39:44.591307 23368 tablet_replica.cc:330] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc: stopping tablet replica
I20260806 06:39:44.591476 23368 raft_consensus.cc:2229] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:39:44.591634 23368 raft_consensus.cc:2258] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:44.592543 23368 ts_tablet_manager.cc:1905] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:44.594113 23368 ts_tablet_manager.cc:1918] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:44.594225 23368 log.cc:1196] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/211fddfcfc5142449991a4149d7656a7
I20260806 06:39:44.594558 23368 ts_tablet_manager.cc:1939] T 211fddfcfc5142449991a4149d7656a7 P affa3a2f74674cc4af30b0de1d6b6fbc: Deleting consensus metadata
I20260806 06:39:44.595177 22654 catalog_manager.cc:4841] TS affa3a2f74674cc4af30b0de1d6b6fbc (127.29.142.130:38061): tablet 211fddfcfc5142449991a4149d7656a7 (table test-table [id=1a22b202afd44b76a8fe70987fabf58a]) successfully deleted
W20260806 06:39:44.596808 22653 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:36553 (error 108)
I20260806 06:39:44.597405 23336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:44.597242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:36553 (local address 127.0.0.1:56888)
0806 06:39:44.597311 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:44.597360 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:36553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 06:39:44.597528 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 22937
W20260806 06:39:44.601882 22653 catalog_manager.cc:4584] TS 0905b6f5f6c045d8afd84b56cb61dff5 (127.29.142.129:36553): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 211fddfcfc5142449991a4149d7656a7: Network error: Client connection negotiation failed: client connection to 127.29.142.129:36553: connect: Connection refused (error 111)
I20260806 06:39:44.612646 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 23131
I20260806 06:39:44.624640 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 22633
2026-08-06T06:39:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (967 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:39:44 dist-test-slave-3kk6 krb5kdc[23377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:39:44 dist-test-slave-3kk6 krb5kdc[23377](info): set up 2 sockets
Aug 06 06:39:44 dist-test-slave-3kk6 krb5kdc[23377](info): commencing operation
krb5kdc: starting...
W20260806 06:39:45.921440 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:39:46 dist-test-slave-3kk6 krb5kdc[23377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:39:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:39:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:39:46.053362 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:35439
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:35439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:46.142740 23400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:46.143009 23400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:46.143051 23400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:46.144240 23400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:46.144320 23400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:46.144345 23400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:46.144366 23400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:46.144397 23400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:46.145941 23400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:35439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:35439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:46.146246 23400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:46.146497 23400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:46.151973 23408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:46.151957 23412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:46.152215 23400 server_base.cc:1034] running on GCE node
W20260806 06:39:46.151945 23410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:46.152542 23400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:46.152915 23400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:46.154094 23400 hybrid_clock.cc:648] HybridClock initialized: now 1785998386154067 us; error 47 us; skew 500 ppm
Aug 06 06:39:46 dist-test-slave-3kk6 krb5kdc[23377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998386, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:46.156852 23400 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:39:46.157323 23400 webserver.cc:460] Webserver started at http://127.29.142.190:32785/ using document root <none> and password file <none>
I20260806 06:39:46.157562 23400 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:46.157634 23400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:46.157802 23400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:46.159090 23400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "e6d68605e07340dda0f9fd92e0cb9d59"
format_stamp: "Formatted at 2026-08-06 06:39:46 on dist-test-slave-3kk6"
server_key: "41743b69c4cc362244fdb5468bb8443f"
server_key_iv: "2fb4ac5d46822036b4098212b8d94f9b"
server_key_version: "encryptionkey@0"
I20260806 06:39:46.159528 23400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "e6d68605e07340dda0f9fd92e0cb9d59"
format_stamp: "Formatted at 2026-08-06 06:39:46 on dist-test-slave-3kk6"
server_key: "41743b69c4cc362244fdb5468bb8443f"
server_key_iv: "2fb4ac5d46822036b4098212b8d94f9b"
server_key_version: "encryptionkey@0"
I20260806 06:39:46.161335 23400 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 06:39:46.162220 23420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:46.162503 23400 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:46.162593 23400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "e6d68605e07340dda0f9fd92e0cb9d59"
format_stamp: "Formatted at 2026-08-06 06:39:46 on dist-test-slave-3kk6"
server_key: "41743b69c4cc362244fdb5468bb8443f"
server_key_iv: "2fb4ac5d46822036b4098212b8d94f9b"
server_key_version: "encryptionkey@0"
I20260806 06:39:46.162734 23400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:46.190064 23400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:46.192373 23400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:46.192541 23400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:46.197175 23400 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:35439
I20260806 06:39:46.197222 23507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:35439 every 8 connection(s)
I20260806 06:39:46.197615 23400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:46.198578 23508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:46.199337 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 23400
I20260806 06:39:46.199461 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:39:46.199687 30266 external_mini_cluster.cc:1428] Setting key 6b5e1143eee61c086ed79f6ca1926e15
I20260806 06:39:46.202159 23508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59: Bootstrap starting.
I20260806 06:39:46.202855 23508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:46.203167 23508 log.cc:824] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:46.203974 23508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59: No bootstrap required, opened a new log
I20260806 06:39:46.206701 23508 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d68605e07340dda0f9fd92e0cb9d59" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35439 } }
I20260806 06:39:46.206869 23508 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:46.206894 23508 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6d68605e07340dda0f9fd92e0cb9d59, State: Initialized, Role: FOLLOWER
I20260806 06:39:46.207048 23508 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d68605e07340dda0f9fd92e0cb9d59" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35439 } }
I20260806 06:39:46.207124 23508 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:46.207149 23508 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:46.207228 23508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 06:39:46 dist-test-slave-3kk6 krb5kdc[23377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:46.207929 23508 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d68605e07340dda0f9fd92e0cb9d59" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35439 } }
I20260806 06:39:46.208053 23508 leader_election.cc:302] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6d68605e07340dda0f9fd92e0cb9d59; no voters:
I20260806 06:39:46.208309 23508 leader_election.cc:288] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:39:46.208343 23514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:46.208470 23514 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [term 1 LEADER]: Becoming Leader. State: Replica: e6d68605e07340dda0f9fd92e0cb9d59, State: Running, Role: LEADER
I20260806 06:39:46.208564 23514 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d68605e07340dda0f9fd92e0cb9d59" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35439 } }
I20260806 06:39:46.208635 23508 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:46.208901 23514 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6d68605e07340dda0f9fd92e0cb9d59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d68605e07340dda0f9fd92e0cb9d59" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35439 } } }
I20260806 06:39:46.208966 23514 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:46.209100 23514 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6d68605e07340dda0f9fd92e0cb9d59. Latest consensus state: current_term: 1 leader_uuid: "e6d68605e07340dda0f9fd92e0cb9d59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6d68605e07340dda0f9fd92e0cb9d59" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35439 } } }
I20260806 06:39:46.209174 23514 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59 [sys.catalog]: This master's current role is: LEADER
W20260806 06:39:46.210574 23537 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:39:46.210654 23537 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:39:46.210752 23519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:46.210951 23519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:39:46.212210 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.200338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34724 (local address 127.29.142.190:35439)
0806 06:39:46.200524 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:46.200534 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:46.200561 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:39:46.200987 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:46.200994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:46.201044 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:46.201172 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:46.202072 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.203168 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.204253 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.204669 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.204758 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.207948 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:46.207953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:46.207954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:46.207968 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:46.210256 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:46.210791 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:46.210795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:46.210795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:46.210915 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:46.211173 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:46.211174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:46.211175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:46.211500 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:46.211685 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:46.211837 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:39:46.211998 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260806 06:39:46.212853 23519 catalog_manager.cc:1347] Generated new cluster ID: 55a3c97e86504af79603ae4a209b47ab
I20260806 06:39:46.212927 23519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:39:46.230245 23519 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:39:46.231158 23519 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:46.239387 23519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59: Generated new TSK 0
I20260806 06:39:46.239638 23519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:39:46.267766 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35439
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:46.355458 23544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:46.355680 23544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:46.355706 23544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:46.356884 23544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:46.356931 23544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:46.356958 23544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:39:46.358556 23544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:46.358860 23544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:46.359125 23544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:46.364554 23550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:46.364548 23549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:46.364586 23544 server_base.cc:1034] running on GCE node
W20260806 06:39:46.364544 23552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:46.365074 23544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:46.365411 23544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:46.366590 23544 hybrid_clock.cc:648] HybridClock initialized: now 1785998386366568 us; error 49 us; skew 500 ppm
Aug 06 06:39:46 dist-test-slave-3kk6 krb5kdc[23377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998386, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:46.369357 23544 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:39:46.369865 23544 webserver.cc:460] Webserver started at http://127.29.142.129:38829/ using document root <none> and password file <none>
I20260806 06:39:46.370131 23544 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:46.370203 23544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:46.370369 23544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:46.371531 23544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "bbadfd70bcc04389a750577a497f9206"
format_stamp: "Formatted at 2026-08-06 06:39:46 on dist-test-slave-3kk6"
server_key: "303696db158cafb442d38e45e3871b45"
server_key_iv: "894200b79db3666477535a32f73e979b"
server_key_version: "encryptionkey@0"
I20260806 06:39:46.371934 23544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "bbadfd70bcc04389a750577a497f9206"
format_stamp: "Formatted at 2026-08-06 06:39:46 on dist-test-slave-3kk6"
server_key: "303696db158cafb442d38e45e3871b45"
server_key_iv: "894200b79db3666477535a32f73e979b"
server_key_version: "encryptionkey@0"
I20260806 06:39:46.373306 23544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:39:46.374068 23560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:46.374248 23544 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:39:46.374358 23544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "bbadfd70bcc04389a750577a497f9206"
format_stamp: "Formatted at 2026-08-06 06:39:46 on dist-test-slave-3kk6"
server_key: "303696db158cafb442d38e45e3871b45"
server_key_iv: "894200b79db3666477535a32f73e979b"
server_key_version: "encryptionkey@0"
I20260806 06:39:46.374444 23544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:46.383947 23544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:46.385878 23544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:46.386046 23544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:46.386296 23544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:46.386615 23544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:46.386723 23544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:46.386790 23544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:46.386824 23544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:46.392889 23544 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:35029
I20260806 06:39:46.392935 23715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:35029 every 8 connection(s)
I20260806 06:39:46.393280 23544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:39:46 dist-test-slave-3kk6 krb5kdc[23377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998386, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:46.403155 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.394044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:52523 (local address 127.29.142.190:35439)
0806 06:39:46.394097 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:46.394099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:46.394334 (+ 235us) server_negotiation.cc:407] Connection header received
0806 06:39:46.394980 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:46.394982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:46.394994 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:46.395053 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:46.396051 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.396562 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.397620 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.397840 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.397920 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.400351 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:46.400353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:46.400353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:46.400369 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:46.401629 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:46.402084 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:46.402085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:46.402086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:46.402150 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:46.402431 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:46.402432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:46.402433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:46.402559 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:46.402641 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:46.402959 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 06:39:46.403055 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:39:46.403354 23719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.394015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:35439 (local address 127.29.142.129:52523)
0806 06:39:46.394202 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:46.394240 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:46.394867 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:46.395125 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:46.395130 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:46.395741 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:46.395940 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.395944 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.396674 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.396676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.397522 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.397524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.397929 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.397930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.398067 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.398619 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:46.398629 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:46.400240 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:46.401757 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:46.401765 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:46.401767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:46.402000 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:46.402224 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:46.402225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:46.402225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:46.402355 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:46.402631 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:46.402639 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:46.402813 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 06:39:46.402933 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 06:39:46.403097 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260806 06:39:46.403580 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 23544
I20260806 06:39:46.403707 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:39:46.403995 30266 external_mini_cluster.cc:1428] Setting key 1a1cbcf13fa6859e68f9a46fc9ad316f
I20260806 06:39:46.404095 23717 heartbeater.cc:347] Connected to a master server at 127.29.142.190:35439
I20260806 06:39:46.404208 23717 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:46.404424 23717 heartbeater.cc:511] Master 127.29.142.190:35439 requested a full tablet report, sending...
I20260806 06:39:46.405002 23442 ts_manager.cc:194] Registered new tserver with Master: bbadfd70bcc04389a750577a497f9206 (127.29.142.129:35029)
I20260806 06:39:46.405609 23442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:52523
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:39:46.433121 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35439
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:46.519389 23725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:46.519623 23725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:46.519649 23725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:46.520749 23725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:46.520795 23725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:46.520823 23725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:39:46.522244 23725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:46.522446 23725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:46.522644 23725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:46.528021 23733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:46.528018 23738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:46.528016 23734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:46.528311 23725 server_base.cc:1034] running on GCE node
I20260806 06:39:46.528522 23725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:46.528746 23725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:46.529928 23725 hybrid_clock.cc:648] HybridClock initialized: now 1785998386529916 us; error 42 us; skew 500 ppm
Aug 06 06:39:46 dist-test-slave-3kk6 krb5kdc[23377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998386, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:46.532750 23725 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:39:46.533245 23725 webserver.cc:460] Webserver started at http://127.29.142.130:34777/ using document root <none> and password file <none>
I20260806 06:39:46.533505 23725 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:46.533582 23725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:46.533751 23725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:46.535004 23725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "eeaa1857a1954a79bcec644a6111951f"
format_stamp: "Formatted at 2026-08-06 06:39:46 on dist-test-slave-3kk6"
server_key: "00948c2276b877f0573a35930506ce15"
server_key_iv: "aa31c8e5afaa38e0fbc9ba8eb1c3ce28"
server_key_version: "encryptionkey@0"
I20260806 06:39:46.535413 23725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "eeaa1857a1954a79bcec644a6111951f"
format_stamp: "Formatted at 2026-08-06 06:39:46 on dist-test-slave-3kk6"
server_key: "00948c2276b877f0573a35930506ce15"
server_key_iv: "aa31c8e5afaa38e0fbc9ba8eb1c3ce28"
server_key_version: "encryptionkey@0"
I20260806 06:39:46.536875 23725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:39:46.537750 23750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:46.537949 23725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:46.538094 23725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "eeaa1857a1954a79bcec644a6111951f"
format_stamp: "Formatted at 2026-08-06 06:39:46 on dist-test-slave-3kk6"
server_key: "00948c2276b877f0573a35930506ce15"
server_key_iv: "aa31c8e5afaa38e0fbc9ba8eb1c3ce28"
server_key_version: "encryptionkey@0"
I20260806 06:39:46.538182 23725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:46.551412 23725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:46.553645 23725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:46.553838 23725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:46.554105 23725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:46.554464 23725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:46.554529 23725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:46.554574 23725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:46.554605 23725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:46.560832 23725 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:38573
I20260806 06:39:46.560917 23909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:38573 every 8 connection(s)
I20260806 06:39:46.561285 23725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:39:46 dist-test-slave-3kk6 krb5kdc[23377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998386, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:46.569011 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 23725
I20260806 06:39:46.569134 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:39:46.569378 30266 external_mini_cluster.cc:1428] Setting key 2abea6085c925dda7d101fb92f2ce43f
I20260806 06:39:46.572228 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.562122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:35129 (local address 127.29.142.190:35439)
0806 06:39:46.562176 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:46.562178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:46.562404 (+ 226us) server_negotiation.cc:407] Connection header received
0806 06:39:46.563028 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:46.563030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:46.563043 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:46.563112 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:46.564142 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.564686 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.565827 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.566084 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.566166 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.568529 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:46.568530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:46.568531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:46.568538 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:46.570248 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:46.570810 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:46.570810 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:46.570811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:46.571025 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:46.571408 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:46.571409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:46.571409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:46.571530 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:46.571584 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:46.571957 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 06:39:46.572097 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:39:46.572429 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.562075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:35439 (local address 127.29.142.130:35129)
0806 06:39:46.562261 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:46.562306 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:46.562888 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:46.563189 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:46.563193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:46.563826 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:46.564036 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.564041 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.564783 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.564784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.565701 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.565703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.566199 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.566201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.566385 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.566998 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:46.567007 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:46.568436 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:46.570330 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:46.570339 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:46.570342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:46.570715 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:46.571134 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:46.571136 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:46.571136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:46.571307 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:46.571600 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:46.571607 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:46.571802 (+ 195us) client_negotiation.cc:769] Sending connection context
0806 06:39:46.571920 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 06:39:46.572123 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
I20260806 06:39:46.572979 23911 heartbeater.cc:347] Connected to a master server at 127.29.142.190:35439
I20260806 06:39:46.573097 23911 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:46.573307 23911 heartbeater.cc:511] Master 127.29.142.190:35439 requested a full tablet report, sending...
I20260806 06:39:46.573762 23442 ts_manager.cc:194] Registered new tserver with Master: eeaa1857a1954a79bcec644a6111951f (127.29.142.130:38573)
I20260806 06:39:46.574148 23442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:35129
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:39:46.598322 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35439
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:40555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:46.685119 23917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:46.685400 23917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:46.685443 23917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:46.686637 23917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:46.686754 23917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:46.686801 23917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:39:46.688319 23917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:40555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:46.688618 23917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:46.688872 23917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:46.694939 23922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:46.694909 23923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:46.694927 23917 server_base.cc:1034] running on GCE node
W20260806 06:39:46.694909 23926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:46.695395 23917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:46.695667 23917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:46.696830 23917 hybrid_clock.cc:648] HybridClock initialized: now 1785998386696816 us; error 32 us; skew 500 ppm
Aug 06 06:39:46 dist-test-slave-3kk6 krb5kdc[23377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998386, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:46.699797 23917 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:39:46.700368 23917 webserver.cc:460] Webserver started at http://127.29.142.131:39365/ using document root <none> and password file <none>
I20260806 06:39:46.700613 23917 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:46.700663 23917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:46.700865 23917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:46.702034 23917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "a7edac74448647ecb8ff7dd1c77ffce5"
format_stamp: "Formatted at 2026-08-06 06:39:46 on dist-test-slave-3kk6"
server_key: "53d08cbae55fa1beb68be3f1067a011a"
server_key_iv: "6bbcdbbc0a4d51e9356df15eaaafdd3d"
server_key_version: "encryptionkey@0"
I20260806 06:39:46.702432 23917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "a7edac74448647ecb8ff7dd1c77ffce5"
format_stamp: "Formatted at 2026-08-06 06:39:46 on dist-test-slave-3kk6"
server_key: "53d08cbae55fa1beb68be3f1067a011a"
server_key_iv: "6bbcdbbc0a4d51e9356df15eaaafdd3d"
server_key_version: "encryptionkey@0"
I20260806 06:39:46.704030 23917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:39:46.704895 23935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:46.705089 23917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:46.705206 23917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "a7edac74448647ecb8ff7dd1c77ffce5"
format_stamp: "Formatted at 2026-08-06 06:39:46 on dist-test-slave-3kk6"
server_key: "53d08cbae55fa1beb68be3f1067a011a"
server_key_iv: "6bbcdbbc0a4d51e9356df15eaaafdd3d"
server_key_version: "encryptionkey@0"
I20260806 06:39:46.705291 23917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:46.724406 23917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:46.726804 23917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:46.727020 23917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:46.727340 23917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:46.727720 23917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:46.727784 23917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:46.727836 23917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:46.727864 23917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:46.734797 23917 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:46205
I20260806 06:39:46.734877 24089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:46205 every 8 connection(s)
I20260806 06:39:46.735289 23917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:39:46 dist-test-slave-3kk6 krb5kdc[23377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998386, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:46.744853 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 23917
I20260806 06:39:46.744984 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:39:46.745174 30266 external_mini_cluster.cc:1428] Setting key 79faa690cf758b949ca1c9db2c502b30
I20260806 06:39:46.746711 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.736260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:60027 (local address 127.29.142.190:35439)
0806 06:39:46.736328 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:46.736330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:46.736560 (+ 230us) server_negotiation.cc:407] Connection header received
0806 06:39:46.737101 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:46.737102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:46.737112 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:46.737177 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:46.738247 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.738838 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.740144 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.740342 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.740406 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.743238 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:46.743240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:46.743240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:46.743248 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:46.744757 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:46.745315 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:46.745318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:46.745318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:46.745412 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:46.745868 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:46.745870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:46.745870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:46.746031 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:46.746122 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:46.746490 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 06:39:46.746582 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:39:46.746989 24095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.736216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:35439 (local address 127.29.142.131:60027)
0806 06:39:46.736404 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:46.736454 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:46.737009 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:46.737243 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:46.737247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:46.737915 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:46.738140 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.738146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.738934 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.738935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.740055 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.740058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.740431 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.740432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.740609 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.741341 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:46.741353 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:46.743135 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:46.744878 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:46.744887 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:46.744889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:46.745205 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:46.745587 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:46.745589 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:46.745589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:46.745762 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:46.746113 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:46.746121 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:46.746305 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 06:39:46.746462 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 06:39:46.746685 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260806 06:39:46.747478 24090 heartbeater.cc:347] Connected to a master server at 127.29.142.190:35439
I20260806 06:39:46.747581 24090 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:46.747750 24090 heartbeater.cc:511] Master 127.29.142.190:35439 requested a full tablet report, sending...
I20260806 06:39:46.748188 23442 ts_manager.cc:194] Registered new tserver with Master: a7edac74448647ecb8ff7dd1c77ffce5 (127.29.142.131:46205)
I20260806 06:39:46.748637 23442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:60027
I20260806 06:39:46.757529 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:39:46.766243 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.759176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.29.142.190:35439)
0806 06:39:46.759250 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:46.759252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:46.759382 (+ 130us) server_negotiation.cc:407] Connection header received
0806 06:39:46.759444 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:46.759445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:46.759456 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:46.759508 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:46.760172 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.760679 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.761587 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.761800 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.761861 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.763397 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:46.763399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:46.763399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:46.763406 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:46.764938 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:46.765430 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:46.765431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:46.765431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:46.765489 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:46.765676 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:46.765677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:46.765677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:46.765844 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:46.765930 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:46.766035 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:39:46.766119 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:39:46.768211 23442 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:39:46.769860 23442 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:39:46.776925 24110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.772485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:35029 (local address 127.0.0.1:34322)
0806 06:39:46.772717 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:46.772794 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:46.772873 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:46.773270 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:46.773274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:46.773276 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:46.773380 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.773383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.774445 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.774447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.775736 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.775739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.776528 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.776529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.776611 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.776626 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:39:46.776733 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 06:39:46.776793 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":84,"threads_started":1}
I20260806 06:39:46.776937 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.772910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34322 (local address 127.29.142.129:35029)
0806 06:39:46.773077 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:46.773082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:46.773093 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:39:46.773138 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:46.773140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:46.773158 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:46.773469 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:46.773550 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.774332 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.775841 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.776435 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.776532 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.776595 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:46.776753 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 06:39:46.776793 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260806 06:39:46.777261 24115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.772844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46916 (local address 127.29.142.131:46205)
0806 06:39:46.773040 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:46.773046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:46.773178 (+ 132us) server_negotiation.cc:407] Connection header received
0806 06:39:46.773237 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:46.773239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:46.773257 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:46.773332 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:46.773754 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.774837 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.776062 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.776818 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.776899 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.776958 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:46.777134 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 06:39:46.777184 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":116,"threads_started":1}
I20260806 06:39:46.777184 24113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.772532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:38573 (local address 127.0.0.1:57864)
0806 06:39:46.773063 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:46.773073 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:46.773134 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:46.773451 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:46.773452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:46.773455 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:46.773524 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.773526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.774944 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.774945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.776005 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.776007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.776928 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.776929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.777008 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.777012 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:39:46.777043 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 06:39:46.777091 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":22,"thread_start_us":121,"threads_started":1}
I20260806 06:39:46.777537 24112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.772678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57864 (local address 127.29.142.130:38573)
0806 06:39:46.773292 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:46.773297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:46.773308 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:39:46.773347 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:46.773350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:46.773364 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:46.773438 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:46.773630 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.774802 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.776118 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.776827 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.777281 (+ 454us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.777349 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:46.777405 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 06:39:46.777448 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":101,"threads_started":1}
I20260806 06:39:46.777269 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.772724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:46205 (local address 127.0.0.1:46916)
0806 06:39:46.773094 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:46.773105 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:46.773167 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:46.773612 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:46.773613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:46.773615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:46.773670 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.773672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.774948 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.774949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.775966 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.775969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.776989 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.776990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.777048 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.777050 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:39:46.777114 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:39:46.777147 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":24,"thread_start_us":92,"threads_started":1}
I20260806 06:39:46.778363 23621 tablet_service.cc:1467] Processing CreateTablet for tablet 45f7de463193413ead6c0be98380c94d (DEFAULT_TABLE table=test-table [id=a87a9f35fcfe4b68872c8fb5da685ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:46.778609 24003 tablet_service.cc:1467] Processing CreateTablet for tablet 45f7de463193413ead6c0be98380c94d (DEFAULT_TABLE table=test-table [id=a87a9f35fcfe4b68872c8fb5da685ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:46.778808 23621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45f7de463193413ead6c0be98380c94d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:46.779044 24003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45f7de463193413ead6c0be98380c94d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:46.779120 23818 tablet_service.cc:1467] Processing CreateTablet for tablet 45f7de463193413ead6c0be98380c94d (DEFAULT_TABLE table=test-table [id=a87a9f35fcfe4b68872c8fb5da685ba4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:46.779476 23818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45f7de463193413ead6c0be98380c94d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:46.781808 24117 tablet_bootstrap.cc:492] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5: Bootstrap starting.
I20260806 06:39:46.782552 24118 tablet_bootstrap.cc:492] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206: Bootstrap starting.
I20260806 06:39:46.782629 24117 tablet_bootstrap.cc:654] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:46.783344 24118 tablet_bootstrap.cc:654] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:46.783519 24117 log.cc:824] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:46.783947 24119 tablet_bootstrap.cc:492] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f: Bootstrap starting.
I20260806 06:39:46.783995 24118 log.cc:824] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:46.784439 24117 tablet_bootstrap.cc:492] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5: No bootstrap required, opened a new log
I20260806 06:39:46.784524 24117 ts_tablet_manager.cc:1397] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:39:46.784602 24119 tablet_bootstrap.cc:654] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:46.784811 24118 tablet_bootstrap.cc:492] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206: No bootstrap required, opened a new log
I20260806 06:39:46.784895 24118 ts_tablet_manager.cc:1397] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:39:46.785207 24119 log.cc:824] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:46.786042 24119 tablet_bootstrap.cc:492] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f: No bootstrap required, opened a new log
I20260806 06:39:46.786118 24119 ts_tablet_manager.cc:1397] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:39:46.786520 24117 raft_consensus.cc:348] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbadfd70bcc04389a750577a497f9206" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35029 } } peers { permanent_uuid: "a7edac74448647ecb8ff7dd1c77ffce5" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46205 } } peers { permanent_uuid: "eeaa1857a1954a79bcec644a6111951f" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38573 } }
I20260806 06:39:46.786757 24117 raft_consensus.cc:374] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:46.786813 24117 raft_consensus.cc:729] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7edac74448647ecb8ff7dd1c77ffce5, State: Initialized, Role: FOLLOWER
I20260806 06:39:46.786968 24117 consensus_queue.cc:260] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbadfd70bcc04389a750577a497f9206" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35029 } } peers { permanent_uuid: "a7edac74448647ecb8ff7dd1c77ffce5" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46205 } } peers { permanent_uuid: "eeaa1857a1954a79bcec644a6111951f" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38573 } }
I20260806 06:39:46.787029 24118 raft_consensus.cc:348] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbadfd70bcc04389a750577a497f9206" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35029 } } peers { permanent_uuid: "a7edac74448647ecb8ff7dd1c77ffce5" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46205 } } peers { permanent_uuid: "eeaa1857a1954a79bcec644a6111951f" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38573 } }
I20260806 06:39:46.787225 24118 raft_consensus.cc:374] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:46.787281 24118 raft_consensus.cc:729] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbadfd70bcc04389a750577a497f9206, State: Initialized, Role: FOLLOWER
I20260806 06:39:46.787300 24117 ts_tablet_manager.cc:1428] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 06:39:46.787403 24118 consensus_queue.cc:260] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbadfd70bcc04389a750577a497f9206" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35029 } } peers { permanent_uuid: "a7edac74448647ecb8ff7dd1c77ffce5" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46205 } } peers { permanent_uuid: "eeaa1857a1954a79bcec644a6111951f" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38573 } }
I20260806 06:39:46.787468 24090 heartbeater.cc:503] Master 127.29.142.190:35439 was elected leader, sending a full tablet report...
I20260806 06:39:46.787704 24118 ts_tablet_manager.cc:1428] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:39:46.787842 23717 heartbeater.cc:503] Master 127.29.142.190:35439 was elected leader, sending a full tablet report...
I20260806 06:39:46.788250 24119 raft_consensus.cc:348] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbadfd70bcc04389a750577a497f9206" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35029 } } peers { permanent_uuid: "a7edac74448647ecb8ff7dd1c77ffce5" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46205 } } peers { permanent_uuid: "eeaa1857a1954a79bcec644a6111951f" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38573 } }
I20260806 06:39:46.788429 24119 raft_consensus.cc:374] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:46.788473 24119 raft_consensus.cc:729] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeaa1857a1954a79bcec644a6111951f, State: Initialized, Role: FOLLOWER
I20260806 06:39:46.788589 24119 consensus_queue.cc:260] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbadfd70bcc04389a750577a497f9206" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35029 } } peers { permanent_uuid: "a7edac74448647ecb8ff7dd1c77ffce5" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46205 } } peers { permanent_uuid: "eeaa1857a1954a79bcec644a6111951f" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38573 } }
I20260806 06:39:46.788913 24119 ts_tablet_manager.cc:1428] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:39:46.788923 23911 heartbeater.cc:503] Master 127.29.142.190:35439 was elected leader, sending a full tablet report...
W20260806 06:39:46.811883 23913 tablet.cc:2369] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:39:46.894060 23718 tablet.cc:2369] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:39:46.907328 24127 raft_consensus.cc:482] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:39:46.907481 24127 raft_consensus.cc:504] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbadfd70bcc04389a750577a497f9206" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35029 } } peers { permanent_uuid: "a7edac74448647ecb8ff7dd1c77ffce5" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46205 } } peers { permanent_uuid: "eeaa1857a1954a79bcec644a6111951f" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38573 } }
I20260806 06:39:46.907989 24127 leader_election.cc:288] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbadfd70bcc04389a750577a497f9206 (127.29.142.129:35029), a7edac74448647ecb8ff7dd1c77ffce5 (127.29.142.131:46205)
I20260806 06:39:46.911258 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.908252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:35029 (local address 127.29.142.130:34335)
0806 06:39:46.908321 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:46.908337 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:46.908394 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:46.908633 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:46.908635 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:46.908637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:46.908721 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.908723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.909473 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.909475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.910320 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.910322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.910961 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.910962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.911059 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.911062 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:39:46.911129 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:39:46.911175 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":2}
I20260806 06:39:46.911258 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.908382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:34335 (local address 127.29.142.129:35029)
0806 06:39:46.908435 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:46.908437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:46.908450 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:39:46.908488 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:46.908489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:46.908500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:46.908582 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:46.908817 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.909359 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.910432 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.910871 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.910962 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.910972 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:46.911149 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 06:39:46.911190 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:39:46.911458 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.908413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:46205 (local address 127.29.142.130:58965)
0806 06:39:46.908615 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:46.908626 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:46.908717 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:46.908853 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:46.908853 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:46.908855 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:46.908939 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.908941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.909764 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.909765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.910499 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:46.910501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.911178 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.911178 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:46.911256 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.911260 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:39:46.911330 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:39:46.911371 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":1,"thread_start_us":147,"threads_started":1}
I20260806 06:39:46.911444 24115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.908535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:58965 (local address 127.29.142.131:46205)
0806 06:39:46.908585 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:46.908587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:46.908720 (+ 133us) server_negotiation.cc:407] Connection header received
0806 06:39:46.908793 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:46.908794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:46.908804 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:46.908852 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:46.909031 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.909676 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.910621 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.911072 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.911169 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.911178 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:46.911336 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 06:39:46.911378 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:39:46.911876 24029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f7de463193413ead6c0be98380c94d" candidate_uuid: "eeaa1857a1954a79bcec644a6111951f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7edac74448647ecb8ff7dd1c77ffce5" is_pre_election: true
I20260806 06:39:46.911895 23655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f7de463193413ead6c0be98380c94d" candidate_uuid: "eeaa1857a1954a79bcec644a6111951f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbadfd70bcc04389a750577a497f9206" is_pre_election: true
I20260806 06:39:46.912045 24029 raft_consensus.cc:2454] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eeaa1857a1954a79bcec644a6111951f in term 0.
I20260806 06:39:46.912079 23655 raft_consensus.cc:2454] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eeaa1857a1954a79bcec644a6111951f in term 0.
I20260806 06:39:46.912359 23751 leader_election.cc:302] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7edac74448647ecb8ff7dd1c77ffce5, eeaa1857a1954a79bcec644a6111951f; no voters:
I20260806 06:39:46.912504 24127 raft_consensus.cc:2781] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:39:46.912568 24127 raft_consensus.cc:482] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:39:46.912593 24127 raft_consensus.cc:3037] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:46.913371 24127 raft_consensus.cc:504] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbadfd70bcc04389a750577a497f9206" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35029 } } peers { permanent_uuid: "a7edac74448647ecb8ff7dd1c77ffce5" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46205 } } peers { permanent_uuid: "eeaa1857a1954a79bcec644a6111951f" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38573 } }
I20260806 06:39:46.913530 24127 leader_election.cc:288] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [CANDIDATE]: Term 1 election: Requested vote from peers bbadfd70bcc04389a750577a497f9206 (127.29.142.129:35029), a7edac74448647ecb8ff7dd1c77ffce5 (127.29.142.131:46205)
I20260806 06:39:46.913753 24029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f7de463193413ead6c0be98380c94d" candidate_uuid: "eeaa1857a1954a79bcec644a6111951f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7edac74448647ecb8ff7dd1c77ffce5"
I20260806 06:39:46.913756 23655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45f7de463193413ead6c0be98380c94d" candidate_uuid: "eeaa1857a1954a79bcec644a6111951f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbadfd70bcc04389a750577a497f9206"
I20260806 06:39:46.913857 23655 raft_consensus.cc:3037] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:46.913857 24029 raft_consensus.cc:3037] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:46.914942 24029 raft_consensus.cc:2454] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eeaa1857a1954a79bcec644a6111951f in term 1.
I20260806 06:39:46.914942 23655 raft_consensus.cc:2454] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eeaa1857a1954a79bcec644a6111951f in term 1.
I20260806 06:39:46.915190 23751 leader_election.cc:302] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7edac74448647ecb8ff7dd1c77ffce5, eeaa1857a1954a79bcec644a6111951f; no voters:
I20260806 06:39:46.915302 24127 raft_consensus.cc:2781] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:46.915499 24127 raft_consensus.cc:686] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 1 LEADER]: Becoming Leader. State: Replica: eeaa1857a1954a79bcec644a6111951f, State: Running, Role: LEADER
I20260806 06:39:46.915614 24127 consensus_queue.cc:237] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbadfd70bcc04389a750577a497f9206" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35029 } } peers { permanent_uuid: "a7edac74448647ecb8ff7dd1c77ffce5" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46205 } } peers { permanent_uuid: "eeaa1857a1954a79bcec644a6111951f" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38573 } }
I20260806 06:39:46.916613 23441 catalog_manager.cc:5495] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f reported cstate change: term changed from 0 to 1, leader changed from <none> to eeaa1857a1954a79bcec644a6111951f (127.29.142.130). New cstate: current_term: 1 leader_uuid: "eeaa1857a1954a79bcec644a6111951f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbadfd70bcc04389a750577a497f9206" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7edac74448647ecb8ff7dd1c77ffce5" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eeaa1857a1954a79bcec644a6111951f" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38573 } health_report { overall_health: HEALTHY } } }
I20260806 06:39:46.925945 24112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.923558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57876 (local address 127.29.142.130:38573)
0806 06:39:46.923621 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:46.923623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:46.923637 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:39:46.923674 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:46.923675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:46.923693 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:46.923790 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:39:46.923907 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.924377 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.925225 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:46.925412 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:46.925472 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:46.925593 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:39:46.925692 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:39:46.925795 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:46.925840 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 06:39:46.925853 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:39:46.927973 23655 raft_consensus.cc:1261] T 45f7de463193413ead6c0be98380c94d P bbadfd70bcc04389a750577a497f9206 [term 1 FOLLOWER]: Refusing update from remote peer eeaa1857a1954a79bcec644a6111951f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:46.927971 24029 raft_consensus.cc:1261] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5 [term 1 FOLLOWER]: Refusing update from remote peer eeaa1857a1954a79bcec644a6111951f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:46.928426 24131 consensus_queue.cc:1035] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbadfd70bcc04389a750577a497f9206" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:46.928577 24127 consensus_queue.cc:1035] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7edac74448647ecb8ff7dd1c77ffce5" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 46205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:46.931867 24141 mvcc.cc:205] Tried to move back new op lower bound from 7315449392853966848 to 7315449392807174144. Current Snapshot: MvccSnapshot[applied={T|T < 7315449392853966848 or (T in {7315449392853966848})}]
I20260806 06:39:46.932446 24137 mvcc.cc:205] Tried to move back new op lower bound from 7315449392853966848 to 7315449392807174144. Current Snapshot: MvccSnapshot[applied={T|T < 7315449392853966848 or (T in {7315449392853966848})}]
I20260806 06:39:46.934140 23442 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:46.934262 23442 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:46.935137 23442 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6d68605e07340dda0f9fd92e0cb9d59: Sending DeleteTablet for 3 replicas of tablet 45f7de463193413ead6c0be98380c94d
I20260806 06:39:46.935554 23621 tablet_service.cc:1514] Processing DeleteTablet for tablet 45f7de463193413ead6c0be98380c94d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:46 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:34322
I20260806 06:39:46.935587 24003 tablet_service.cc:1514] Processing DeleteTablet for tablet 45f7de463193413ead6c0be98380c94d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:46 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:46916
I20260806 06:39:46.935638 23818 tablet_service.cc:1514] Processing DeleteTablet for tablet 45f7de463193413ead6c0be98380c94d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:46 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:57864
I20260806 06:39:46.935822 24149 tablet_replica.cc:330] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5: stopping tablet replica
I20260806 06:39:46.935927 24149 raft_consensus.cc:2229] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:39:46.936046 24149 raft_consensus.cc:2258] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:46.936225 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 23544
I20260806 06:39:46.936523 24149 ts_tablet_manager.cc:1905] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:46.948683 24149 ts_tablet_manager.cc:1918] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:46.948812 24149 log.cc:1196] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/45f7de463193413ead6c0be98380c94d
I20260806 06:39:46.949121 24149 ts_tablet_manager.cc:1939] T 45f7de463193413ead6c0be98380c94d P a7edac74448647ecb8ff7dd1c77ffce5: Deleting consensus metadata
I20260806 06:39:46.949640 23422 catalog_manager.cc:4841] TS a7edac74448647ecb8ff7dd1c77ffce5 (127.29.142.131:46205): tablet 45f7de463193413ead6c0be98380c94d (table test-table [id=a87a9f35fcfe4b68872c8fb5da685ba4]) successfully deleted
I20260806 06:39:46.951351 24151 tablet_replica.cc:330] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f: stopping tablet replica
I20260806 06:39:46.951639 24151 raft_consensus.cc:2229] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:39:46.951920 24151 raft_consensus.cc:2258] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:46.955154 24151 ts_tablet_manager.cc:1905] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:46.956979 24151 ts_tablet_manager.cc:1918] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:46.957072 24151 log.cc:1196] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/45f7de463193413ead6c0be98380c94d
W20260806 06:39:46.957114 23422 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:35029 (error 108)
I20260806 06:39:46.957382 24151 ts_tablet_manager.cc:1939] T 45f7de463193413ead6c0be98380c94d P eeaa1857a1954a79bcec644a6111951f: Deleting consensus metadata
I20260806 06:39:46.957469 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 23725
I20260806 06:39:46.960217 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.960041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:35029 (local address 127.0.0.1:34330)
0806 06:39:46.960112 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:46.960173 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:35029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 06:39:46.960366 23422 catalog_manager.cc:4584] TS bbadfd70bcc04389a750577a497f9206 (127.29.142.129:35029): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 45f7de463193413ead6c0be98380c94d: Network error: Client connection negotiation failed: client connection to 127.29.142.129:35029: connect: Connection refused (error 111)
I20260806 06:39:46.965713 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 23917
I20260806 06:39:46.965755 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:46.965613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:38573 (local address 127.0.0.1:57890)
0806 06:39:46.965681 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:46.965718 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.130:38573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 06:39:46.984170 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 23400
2026-08-06T06:39:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2362 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T06:39:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:39:47Z Disabled control of system clock
I20260806 06:39:47.024351 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:46575
--webserver_interface=127.29.142.190
--webserver_port=0
--builtin_ntp_servers=127.29.142.148:34591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:46575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:39:47.124280 24159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:47.124643 24159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:47.124686 24159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:47.126835 24159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:47.126916 24159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:47.126942 24159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:47.126968 24159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:47.127002 24159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:47.129613 24159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:46575
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:46575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:47.129976 24159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:47.130400 24159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:47.137514 24166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:47.137626 24168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:47.137514 24165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:47.137884 24159 server_base.cc:1034] running on GCE node
I20260806 06:39:47.138154 24159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:47.138487 24159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:47.139679 24159 hybrid_clock.cc:648] HybridClock initialized: now 1785998387139642 us; error 50 us; skew 500 ppm
I20260806 06:39:47.141417 24159 webserver.cc:460] Webserver started at http://127.29.142.190:36569/ using document root <none> and password file <none>
I20260806 06:39:47.141750 24159 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:47.141819 24159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:47.141985 24159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:47.143594 24159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "42c4d440bcc04fe5a9d92844658d7929"
format_stamp: "Formatted at 2026-08-06 06:39:47 on dist-test-slave-3kk6"
server_key: "2fa4d6290ccb2647d1432f7ac9432ba2"
server_key_iv: "d7f401f487f3205c0dfa181f81d70015"
server_key_version: "encryptionkey@0"
I20260806 06:39:47.144057 24159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "42c4d440bcc04fe5a9d92844658d7929"
format_stamp: "Formatted at 2026-08-06 06:39:47 on dist-test-slave-3kk6"
server_key: "2fa4d6290ccb2647d1432f7ac9432ba2"
server_key_iv: "d7f401f487f3205c0dfa181f81d70015"
server_key_version: "encryptionkey@0"
I20260806 06:39:47.146066 24159 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 06:39:47.147169 24178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:47.147370 24159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:39:47.147472 24159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "42c4d440bcc04fe5a9d92844658d7929"
format_stamp: "Formatted at 2026-08-06 06:39:47 on dist-test-slave-3kk6"
server_key: "2fa4d6290ccb2647d1432f7ac9432ba2"
server_key_iv: "d7f401f487f3205c0dfa181f81d70015"
server_key_version: "encryptionkey@0"
I20260806 06:39:47.147555 24159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:47.191764 24159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:47.192162 24159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:47.192346 24159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:47.196632 24159 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:46575
I20260806 06:39:47.196672 24247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:46575 every 8 connection(s)
I20260806 06:39:47.197103 24159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:47.198170 24248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:47.200690 24248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929: Bootstrap starting.
I20260806 06:39:47.201130 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 24159
I20260806 06:39:47.201264 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:39:47.201391 24248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:47.201480 30266 external_mini_cluster.cc:1428] Setting key 058efc0326e10c6dfb690550e3690188
I20260806 06:39:47.201795 24248 log.cc:824] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:47.202769 24248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929: No bootstrap required, opened a new log
I20260806 06:39:47.205479 24248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c4d440bcc04fe5a9d92844658d7929" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46575 } }
I20260806 06:39:47.205652 24248 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:47.205677 24248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c4d440bcc04fe5a9d92844658d7929, State: Initialized, Role: FOLLOWER
I20260806 06:39:47.205839 24248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c4d440bcc04fe5a9d92844658d7929" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46575 } }
I20260806 06:39:47.205928 24248 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:47.205997 24248 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:47.206055 24248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:47.206967 24248 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c4d440bcc04fe5a9d92844658d7929" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46575 } }
I20260806 06:39:47.207108 24248 leader_election.cc:302] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42c4d440bcc04fe5a9d92844658d7929; no voters:
I20260806 06:39:47.207316 24248 leader_election.cc:288] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:39:47.207433 24255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:47.207648 24248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:47.207723 24255 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [term 1 LEADER]: Becoming Leader. State: Replica: 42c4d440bcc04fe5a9d92844658d7929, State: Running, Role: LEADER
I20260806 06:39:47.207854 24255 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c4d440bcc04fe5a9d92844658d7929" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46575 } }
I20260806 06:39:47.209249 24256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42c4d440bcc04fe5a9d92844658d7929" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c4d440bcc04fe5a9d92844658d7929" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46575 } } }
I20260806 06:39:47.209358 24256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:47.209570 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.202101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34748 (local address 127.29.142.190:46575)
0806 06:39:47.202376 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:47.202386 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:47.202424 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:39:47.203079 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:47.203089 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:47.203175 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:47.203366 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:47.203598 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.205182 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.206322 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.207012 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.207157 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.208442 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:47.208447 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:47.208447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:47.208719 (+ 272us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:47.208765 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:47.208782 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:47.208854 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:47.209141 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 06:39:47.209250 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
I20260806 06:39:47.209563 24257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42c4d440bcc04fe5a9d92844658d7929. Latest consensus state: current_term: 1 leader_uuid: "42c4d440bcc04fe5a9d92844658d7929" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c4d440bcc04fe5a9d92844658d7929" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46575 } } }
I20260806 06:39:47.209684 24257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:47.209679 24277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:47.210860 24277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:39:47.212460 24277 catalog_manager.cc:1347] Generated new cluster ID: 438dc5f29a0641fa810858a3435213f1
I20260806 06:39:47.212524 24277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:39:47.237897 24277 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:39:47.238720 24277 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:47.242301 24277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929: Generated new TSK 0
I20260806 06:39:47.242509 24277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:39:47.252460 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:46575
--builtin_ntp_servers=127.29.142.148:34591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:39:47.340159 24287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:47.340431 24287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:47.340473 24287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:47.341683 24287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:47.341753 24287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:47.341959 24287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:39:47.343504 24287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:46575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:47.343808 24287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:47.344144 24287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:47.349432 24295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:47.349550 24300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:47.349545 24296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:47.349670 24287 server_base.cc:1034] running on GCE node
I20260806 06:39:47.349977 24287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:47.350238 24287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:47.351398 24287 hybrid_clock.cc:648] HybridClock initialized: now 1785998387351389 us; error 33 us; skew 500 ppm
I20260806 06:39:47.352751 24287 webserver.cc:460] Webserver started at http://127.29.142.129:36339/ using document root <none> and password file <none>
I20260806 06:39:47.353041 24287 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:47.353116 24287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:47.353262 24287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:47.354404 24287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "bb017e5573ba4713859fe9d6119b31ac"
format_stamp: "Formatted at 2026-08-06 06:39:47 on dist-test-slave-3kk6"
server_key: "52872ac8e377366b74bfdac2ecae661d"
server_key_iv: "2e518da1c1238d97fb54efff8687f4fc"
server_key_version: "encryptionkey@0"
I20260806 06:39:47.354843 24287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "bb017e5573ba4713859fe9d6119b31ac"
format_stamp: "Formatted at 2026-08-06 06:39:47 on dist-test-slave-3kk6"
server_key: "52872ac8e377366b74bfdac2ecae661d"
server_key_iv: "2e518da1c1238d97fb54efff8687f4fc"
server_key_version: "encryptionkey@0"
I20260806 06:39:47.356307 24287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:39:47.357215 24308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:47.357399 24287 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:39:47.357515 24287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "bb017e5573ba4713859fe9d6119b31ac"
format_stamp: "Formatted at 2026-08-06 06:39:47 on dist-test-slave-3kk6"
server_key: "52872ac8e377366b74bfdac2ecae661d"
server_key_iv: "2e518da1c1238d97fb54efff8687f4fc"
server_key_version: "encryptionkey@0"
I20260806 06:39:47.357610 24287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:47.378340 24287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:47.378757 24287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:47.378944 24287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:47.379222 24287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:47.379603 24287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:47.379669 24287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:47.379736 24287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:47.379768 24287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:47.386116 24287 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:38959
I20260806 06:39:47.386176 24469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:38959 every 8 connection(s)
I20260806 06:39:47.386582 24287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:39:47.388659 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 24287
I20260806 06:39:47.388803 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:39:47.389016 30266 external_mini_cluster.cc:1428] Setting key 78ad00e2c95d1c415e95f0e8c6844c37
I20260806 06:39:47.390110 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:46575
--builtin_ntp_servers=127.29.142.148:34591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:39:47.392875 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.387489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:46265 (local address 127.29.142.190:46575)
0806 06:39:47.387578 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:47.387581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:47.387798 (+ 217us) server_negotiation.cc:407] Connection header received
0806 06:39:47.388311 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:47.388312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:47.388323 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:47.388383 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:47.389172 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.389921 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.391080 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.391421 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.391501 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.392597 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:47.392598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:47.392599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:47.392604 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:47.392620 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:47.392624 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:47.392675 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:47.392788 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 06:39:47.392813 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 06:39:47.393151 24472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.387441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:46575 (local address 127.29.142.129:46265)
0806 06:39:47.387656 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:47.387695 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:47.388223 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:47.388439 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:47.388443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:47.388452 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:47.389063 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.389069 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.390017 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.390017 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.390990 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.390993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.391513 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.391514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.391713 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.392416 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:39:47.392417 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:47.392462 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:39:47.392494 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:47.392696 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:47.392703 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:47.392706 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:39:47.392782 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 06:39:47.392910 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
I20260806 06:39:47.393747 24470 heartbeater.cc:347] Connected to a master server at 127.29.142.190:46575
I20260806 06:39:47.393847 24470 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:47.394060 24470 heartbeater.cc:511] Master 127.29.142.190:46575 requested a full tablet report, sending...
I20260806 06:39:47.394680 24201 ts_manager.cc:194] Registered new tserver with Master: bb017e5573ba4713859fe9d6119b31ac (127.29.142.129:38959)
I20260806 06:39:47.395361 24201 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.142.129:46265
W20260806 06:39:47.476549 24473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:47.476819 24473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:47.476857 24473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:47.478021 24473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:47.478096 24473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:47.478139 24473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:39:47.479591 24473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:46575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:47.479902 24473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:47.480247 24473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:39:47.485684 24473 server_base.cc:1034] running on GCE node
W20260806 06:39:47.485651 24483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:47.485646 24480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:47.485646 24479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:47.486142 24473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:47.486367 24473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:47.487524 24473 hybrid_clock.cc:648] HybridClock initialized: now 1785998387487512 us; error 35 us; skew 500 ppm
I20260806 06:39:47.488865 24473 webserver.cc:460] Webserver started at http://127.29.142.130:41157/ using document root <none> and password file <none>
I20260806 06:39:47.489127 24473 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:47.489187 24473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:47.489331 24473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:47.490489 24473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "0dcfedacdf044397a1333c111dca7ba4"
format_stamp: "Formatted at 2026-08-06 06:39:47 on dist-test-slave-3kk6"
server_key: "63251fa9258a5dfe8e11e563e39411e7"
server_key_iv: "b6d0b08f837dedbabf0ccd50e28542d1"
server_key_version: "encryptionkey@0"
I20260806 06:39:47.490933 24473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "0dcfedacdf044397a1333c111dca7ba4"
format_stamp: "Formatted at 2026-08-06 06:39:47 on dist-test-slave-3kk6"
server_key: "63251fa9258a5dfe8e11e563e39411e7"
server_key_iv: "b6d0b08f837dedbabf0ccd50e28542d1"
server_key_version: "encryptionkey@0"
I20260806 06:39:47.492333 24473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:39:47.493124 24490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:47.493319 24473 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:39:47.493420 24473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "0dcfedacdf044397a1333c111dca7ba4"
format_stamp: "Formatted at 2026-08-06 06:39:47 on dist-test-slave-3kk6"
server_key: "63251fa9258a5dfe8e11e563e39411e7"
server_key_iv: "b6d0b08f837dedbabf0ccd50e28542d1"
server_key_version: "encryptionkey@0"
I20260806 06:39:47.493482 24473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:47.509684 24473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:47.510006 24473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:47.510129 24473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:47.510350 24473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:47.510813 24473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:47.510859 24473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:47.510882 24473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:47.510895 24473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:47.517009 24473 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:38157
I20260806 06:39:47.517051 24635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:38157 every 8 connection(s)
I20260806 06:39:47.517439 24473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:39:47.523239 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.518355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:59973 (local address 127.29.142.190:46575)
0806 06:39:47.518472 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:47.518475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:47.518706 (+ 231us) server_negotiation.cc:407] Connection header received
0806 06:39:47.519321 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:47.519322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:47.519337 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:47.519394 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:47.520127 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.520641 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.521667 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.521878 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.521975 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.522933 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:47.522934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:47.522934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:47.522939 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:47.522959 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:47.522964 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:47.523025 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:47.523125 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 06:39:47.523152 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 06:39:47.523486 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.518260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:46575 (local address 127.29.142.130:59973)
0806 06:39:47.518528 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:47.518576 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:47.519186 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:47.519469 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:47.519474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:47.519490 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:47.520016 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.520020 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.520750 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.520751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.521566 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.521568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.522000 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.522001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.522137 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.522776 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:39:47.522777 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:47.522807 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:39:47.522829 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:47.523027 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:47.523034 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:47.523037 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:39:47.523128 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 06:39:47.523234 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":127,"threads_started":1}
I20260806 06:39:47.524048 24637 heartbeater.cc:347] Connected to a master server at 127.29.142.190:46575
I20260806 06:39:47.524137 24637 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:47.524364 24637 heartbeater.cc:511] Master 127.29.142.190:46575 requested a full tablet report, sending...
I20260806 06:39:47.524847 24201 ts_manager.cc:194] Registered new tserver with Master: 0dcfedacdf044397a1333c111dca7ba4 (127.29.142.130:38157)
I20260806 06:39:47.525224 24201 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.142.130:59973
I20260806 06:39:47.526889 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 24473
I20260806 06:39:47.526989 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:39:47.527190 30266 external_mini_cluster.cc:1428] Setting key 490f35830fa077d4a43bcf49c9be3bcd
I20260806 06:39:47.528330 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:46575
--builtin_ntp_servers=127.29.142.148:34591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:39:47.614812 24640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:47.615068 24640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:47.615110 24640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:47.616312 24640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:47.616385 24640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:47.616427 24640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:39:47.617774 24640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:34591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:46575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:47.618070 24640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:47.618403 24640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:47.623975 24648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:47.624018 24649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:47.624032 24651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:47.624004 24640 server_base.cc:1034] running on GCE node
I20260806 06:39:47.624437 24640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:47.624719 24640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:47.625922 24640 hybrid_clock.cc:648] HybridClock initialized: now 1785998387625897 us; error 42 us; skew 500 ppm
I20260806 06:39:47.627343 24640 webserver.cc:460] Webserver started at http://127.29.142.131:42469/ using document root <none> and password file <none>
I20260806 06:39:47.627601 24640 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:47.627673 24640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:47.627828 24640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:47.629014 24640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "bde0c6f198b54b6a96850cc62e3a53e3"
format_stamp: "Formatted at 2026-08-06 06:39:47 on dist-test-slave-3kk6"
server_key: "d41b35b1eb1dcf9b1fd47fbbae4ff22e"
server_key_iv: "4b35a15356fce182edec1be04d706c72"
server_key_version: "encryptionkey@0"
I20260806 06:39:47.629415 24640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "bde0c6f198b54b6a96850cc62e3a53e3"
format_stamp: "Formatted at 2026-08-06 06:39:47 on dist-test-slave-3kk6"
server_key: "d41b35b1eb1dcf9b1fd47fbbae4ff22e"
server_key_iv: "4b35a15356fce182edec1be04d706c72"
server_key_version: "encryptionkey@0"
I20260806 06:39:47.630900 24640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:39:47.631705 24657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:47.631881 24640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:39:47.631989 24640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "bde0c6f198b54b6a96850cc62e3a53e3"
format_stamp: "Formatted at 2026-08-06 06:39:47 on dist-test-slave-3kk6"
server_key: "d41b35b1eb1dcf9b1fd47fbbae4ff22e"
server_key_iv: "4b35a15356fce182edec1be04d706c72"
server_key_version: "encryptionkey@0"
I20260806 06:39:47.632079 24640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:47.646945 24640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:47.647321 24640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:47.647485 24640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:47.647753 24640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:47.648106 24640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:47.648165 24640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:47.648238 24640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:47.648272 24640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:47.654564 24640 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:42607
I20260806 06:39:47.654610 24801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:42607 every 8 connection(s)
I20260806 06:39:47.655069 24640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:39:47.660890 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.655993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:41927 (local address 127.29.142.190:46575)
0806 06:39:47.656086 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:47.656088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:47.656316 (+ 228us) server_negotiation.cc:407] Connection header received
0806 06:39:47.656829 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:47.656830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:47.656841 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:47.656914 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:47.657658 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.658161 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.659255 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.659464 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.659543 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.660467 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:47.660468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:47.660468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:47.660474 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:47.660490 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:47.660494 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:47.660542 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:47.660709 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 06:39:47.660783 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 06:39:47.661068 24804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.655943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:46575 (local address 127.29.142.131:41927)
0806 06:39:47.656171 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:47.656216 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:47.656729 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:47.656999 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:47.657002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:47.657010 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:47.657545 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.657549 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.658268 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.658269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.659167 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.659170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.659557 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.659558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.659672 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.660292 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:39:47.660292 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:47.660323 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:39:47.660347 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:47.660560 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:47.660566 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:47.660568 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:39:47.660629 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:39:47.660719 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":122,"threads_started":1}
I20260806 06:39:47.661545 24802 heartbeater.cc:347] Connected to a master server at 127.29.142.190:46575
I20260806 06:39:47.661650 24802 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:47.661813 24802 heartbeater.cc:511] Master 127.29.142.190:46575 requested a full tablet report, sending...
I20260806 06:39:47.662235 24201 ts_manager.cc:194] Registered new tserver with Master: bde0c6f198b54b6a96850cc62e3a53e3 (127.29.142.131:42607)
I20260806 06:39:47.662587 24201 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.142.131:41927
I20260806 06:39:47.664685 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 24640
I20260806 06:39:47.664824 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:39:47.665043 30266 external_mini_cluster.cc:1428] Setting key fe311f9bc137e5b135fe55918465d804
I20260806 06:39:47.666253 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:39:47.671305 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.668055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34750 (local address 127.29.142.190:46575)
0806 06:39:47.668107 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:47.668110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:47.668224 (+ 114us) server_negotiation.cc:407] Connection header received
0806 06:39:47.668280 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:47.668281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:47.668290 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:47.668341 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:47.668486 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.668972 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.669768 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.669957 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.670022 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.670938 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:47.670939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:47.670940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:47.670944 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:47.670957 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:47.670961 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:47.671045 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:47.671165 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 06:39:47.671177 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:39:47.673239 24201 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:39:47.675063 24201 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:39:47.682145 24821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.677867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47450 (local address 127.29.142.131:42607)
0806 06:39:47.678144 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:47.678147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:47.678156 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:39:47.678208 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:47.678209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:47.678222 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:47.678392 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:47.678600 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.679806 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.681105 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.681807 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.681884 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.681971 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:47.682039 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:39:47.682072 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":96,"threads_started":1}
I20260806 06:39:47.682179 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.677867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48440 (local address 127.29.142.130:38157)
0806 06:39:47.678052 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:47.678056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:47.678149 (+ 93us) server_negotiation.cc:407] Connection header received
0806 06:39:47.678242 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:47.678245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:47.678263 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:47.678341 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:47.678604 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.679424 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.680860 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.681478 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.681519 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.681584 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:47.682046 (+ 462us) server_negotiation.cc:299] Negotiation successful
0806 06:39:47.682089 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
I20260806 06:39:47.682181 24814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.677594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:42607 (local address 127.0.0.1:47450)
0806 06:39:47.677938 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:47.677982 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:47.678059 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:47.678464 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:47.678465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:47.678468 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:47.678516 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.678518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.679912 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.679913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.681032 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.681034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.681889 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.681890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.681952 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.681956 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:39:47.682016 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:39:47.682085 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":11,"spinlock_wait_cycles":384,"thread_start_us":123,"threads_started":1}
I20260806 06:39:47.682406 24815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.677597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:38157 (local address 127.0.0.1:48440)
0806 06:39:47.678050 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:47.678061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:47.678157 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:47.678430 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:47.678433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:47.678436 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:47.678511 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.678513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.679509 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.679509 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.680757 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.680760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.681636 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.681637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.681706 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.681715 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 06:39:47.682277 (+ 562us) client_negotiation.cc:240] Negotiation successful
0806 06:39:47.682326 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":28,"thread_start_us":85,"threads_started":1}
I20260806 06:39:47.682778 24817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.677708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.29.142.129:38959)
0806 06:39:47.677910 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:47.677916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:47.678335 (+ 419us) server_negotiation.cc:407] Connection header received
0806 06:39:47.678478 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:47.678481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:47.678497 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:47.678551 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:47.678811 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.679908 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.681034 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.681797 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.681884 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.681972 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:47.682607 (+ 635us) server_negotiation.cc:299] Negotiation successful
0806 06:39:47.682690 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":115,"threads_started":1}
I20260806 06:39:47.683022 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.677599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:38959 (local address 127.0.0.1:44348)
0806 06:39:47.678265 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:47.678274 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:47.678375 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:47.678679 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:47.678680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:47.678682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:47.678717 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.678719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.679973 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.679974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.680930 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.680933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.682423 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.682424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.682494 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.682499 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:39:47.682898 (+ 399us) client_negotiation.cc:240] Negotiation successful
0806 06:39:47.682946 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260806 06:39:47.683053 24718 tablet_service.cc:1467] Processing CreateTablet for tablet c3a0245b4c8046098a4fdeedb0d36f76 (DEFAULT_TABLE table=test-table [id=88cd5bef576846ada8dc94211ad16d56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:47.683164 24550 tablet_service.cc:1467] Processing CreateTablet for tablet c3a0245b4c8046098a4fdeedb0d36f76 (DEFAULT_TABLE table=test-table [id=88cd5bef576846ada8dc94211ad16d56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:47.683511 24718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3a0245b4c8046098a4fdeedb0d36f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:47.683535 24550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3a0245b4c8046098a4fdeedb0d36f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:47.683681 24373 tablet_service.cc:1467] Processing CreateTablet for tablet c3a0245b4c8046098a4fdeedb0d36f76 (DEFAULT_TABLE table=test-table [id=88cd5bef576846ada8dc94211ad16d56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:47.684015 24373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3a0245b4c8046098a4fdeedb0d36f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:47.686559 24823 tablet_bootstrap.cc:492] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3: Bootstrap starting.
I20260806 06:39:47.687160 24824 tablet_bootstrap.cc:492] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4: Bootstrap starting.
I20260806 06:39:47.687426 24823 tablet_bootstrap.cc:654] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:47.687508 24826 tablet_bootstrap.cc:492] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac: Bootstrap starting.
I20260806 06:39:47.687855 24824 tablet_bootstrap.cc:654] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:47.687860 24823 log.cc:824] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:47.688184 24826 tablet_bootstrap.cc:654] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:47.688185 24824 log.cc:824] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:47.688522 24826 log.cc:824] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:47.688714 24823 tablet_bootstrap.cc:492] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3: No bootstrap required, opened a new log
I20260806 06:39:47.688822 24823 ts_tablet_manager.cc:1397] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:39:47.688987 24824 tablet_bootstrap.cc:492] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4: No bootstrap required, opened a new log
I20260806 06:39:47.689080 24824 ts_tablet_manager.cc:1397] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:39:47.689436 24826 tablet_bootstrap.cc:492] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac: No bootstrap required, opened a new log
I20260806 06:39:47.689517 24826 ts_tablet_manager.cc:1397] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:39:47.691298 24823 raft_consensus.cc:348] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb017e5573ba4713859fe9d6119b31ac" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38959 } } peers { permanent_uuid: "0dcfedacdf044397a1333c111dca7ba4" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38157 } } peers { permanent_uuid: "bde0c6f198b54b6a96850cc62e3a53e3" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 42607 } }
I20260806 06:39:47.691469 24823 raft_consensus.cc:374] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:47.691517 24823 raft_consensus.cc:729] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bde0c6f198b54b6a96850cc62e3a53e3, State: Initialized, Role: FOLLOWER
I20260806 06:39:47.691676 24823 consensus_queue.cc:260] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb017e5573ba4713859fe9d6119b31ac" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38959 } } peers { permanent_uuid: "0dcfedacdf044397a1333c111dca7ba4" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38157 } } peers { permanent_uuid: "bde0c6f198b54b6a96850cc62e3a53e3" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 42607 } }
I20260806 06:39:47.691880 24824 raft_consensus.cc:348] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb017e5573ba4713859fe9d6119b31ac" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38959 } } peers { permanent_uuid: "0dcfedacdf044397a1333c111dca7ba4" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38157 } } peers { permanent_uuid: "bde0c6f198b54b6a96850cc62e3a53e3" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 42607 } }
I20260806 06:39:47.692013 24802 heartbeater.cc:503] Master 127.29.142.190:46575 was elected leader, sending a full tablet report...
I20260806 06:39:47.691990 24823 ts_tablet_manager.cc:1428] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:39:47.692049 24824 raft_consensus.cc:374] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:47.692085 24824 raft_consensus.cc:729] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dcfedacdf044397a1333c111dca7ba4, State: Initialized, Role: FOLLOWER
I20260806 06:39:47.692233 24824 consensus_queue.cc:260] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb017e5573ba4713859fe9d6119b31ac" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38959 } } peers { permanent_uuid: "0dcfedacdf044397a1333c111dca7ba4" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38157 } } peers { permanent_uuid: "bde0c6f198b54b6a96850cc62e3a53e3" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 42607 } }
I20260806 06:39:47.692194 24826 raft_consensus.cc:348] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb017e5573ba4713859fe9d6119b31ac" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38959 } } peers { permanent_uuid: "0dcfedacdf044397a1333c111dca7ba4" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38157 } } peers { permanent_uuid: "bde0c6f198b54b6a96850cc62e3a53e3" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 42607 } }
I20260806 06:39:47.692334 24826 raft_consensus.cc:374] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:47.692375 24826 raft_consensus.cc:729] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb017e5573ba4713859fe9d6119b31ac, State: Initialized, Role: FOLLOWER
I20260806 06:39:47.692508 24824 ts_tablet_manager.cc:1428] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:39:47.692544 24826 consensus_queue.cc:260] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb017e5573ba4713859fe9d6119b31ac" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38959 } } peers { permanent_uuid: "0dcfedacdf044397a1333c111dca7ba4" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38157 } } peers { permanent_uuid: "bde0c6f198b54b6a96850cc62e3a53e3" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 42607 } }
I20260806 06:39:47.692629 24637 heartbeater.cc:503] Master 127.29.142.190:46575 was elected leader, sending a full tablet report...
I20260806 06:39:47.692819 24826 ts_tablet_manager.cc:1428] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:39:47.692945 24470 heartbeater.cc:503] Master 127.29.142.190:46575 was elected leader, sending a full tablet report...
I20260806 06:39:47.718984 24835 raft_consensus.cc:482] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:39:47.719141 24835 raft_consensus.cc:504] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb017e5573ba4713859fe9d6119b31ac" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38959 } } peers { permanent_uuid: "0dcfedacdf044397a1333c111dca7ba4" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38157 } } peers { permanent_uuid: "bde0c6f198b54b6a96850cc62e3a53e3" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 42607 } }
I20260806 06:39:47.719594 24835 leader_election.cc:288] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dcfedacdf044397a1333c111dca7ba4 (127.29.142.130:38157), bde0c6f198b54b6a96850cc62e3a53e3 (127.29.142.131:42607)
I20260806 06:39:47.723862 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.719928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:59089 (local address 127.29.142.130:38157)
0806 06:39:47.719992 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:47.719994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:47.720114 (+ 120us) server_negotiation.cc:407] Connection header received
0806 06:39:47.720148 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:47.720149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:47.720158 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:47.720219 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:47.720371 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.721146 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.722834 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.723417 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.723505 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.723514 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:47.723743 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 06:39:47.723787 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:39:47.723862 24821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.719919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:53061 (local address 127.29.142.131:42607)
0806 06:39:47.719971 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:47.719973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:47.720028 (+ 55us) server_negotiation.cc:407] Connection header received
0806 06:39:47.720069 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:47.720069 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:47.720078 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:47.720158 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:47.720373 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.721124 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.722840 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.723417 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.723505 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.723514 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:47.723743 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 06:39:47.723787 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:39:47.723865 24472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.719821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:42607 (local address 127.29.142.129:53061)
0806 06:39:47.719904 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:47.719919 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:47.719969 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:47.720185 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:47.720186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:47.720188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:47.720284 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.720287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.721250 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.721251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.722703 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.722708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.723533 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.723535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.723634 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.723639 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:39:47.723711 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:39:47.723765 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":120}
I20260806 06:39:47.723871 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.719834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:38157 (local address 127.29.142.129:59089)
0806 06:39:47.720042 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:47.720051 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:47.720094 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:47.720231 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:47.720232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:47.720233 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:47.720296 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.720297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.721250 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.721251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.722734 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:47.722737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.723533 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.723535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:47.723634 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.723639 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:39:47.723711 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:39:47.723784 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":73,"thread_start_us":114,"threads_started":1}
I20260806 06:39:47.724323 24576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3a0245b4c8046098a4fdeedb0d36f76" candidate_uuid: "bb017e5573ba4713859fe9d6119b31ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dcfedacdf044397a1333c111dca7ba4" is_pre_election: true
I20260806 06:39:47.724323 24744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3a0245b4c8046098a4fdeedb0d36f76" candidate_uuid: "bb017e5573ba4713859fe9d6119b31ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde0c6f198b54b6a96850cc62e3a53e3" is_pre_election: true
I20260806 06:39:47.724499 24576 raft_consensus.cc:2454] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb017e5573ba4713859fe9d6119b31ac in term 0.
I20260806 06:39:47.724499 24744 raft_consensus.cc:2454] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb017e5573ba4713859fe9d6119b31ac in term 0.
I20260806 06:39:47.724772 24311 leader_election.cc:302] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb017e5573ba4713859fe9d6119b31ac, bde0c6f198b54b6a96850cc62e3a53e3; no voters:
I20260806 06:39:47.724978 24835 raft_consensus.cc:2781] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:39:47.725067 24835 raft_consensus.cc:482] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:39:47.725104 24835 raft_consensus.cc:3037] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:47.725956 24835 raft_consensus.cc:504] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb017e5573ba4713859fe9d6119b31ac" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38959 } } peers { permanent_uuid: "0dcfedacdf044397a1333c111dca7ba4" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38157 } } peers { permanent_uuid: "bde0c6f198b54b6a96850cc62e3a53e3" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 42607 } }
I20260806 06:39:47.726130 24835 leader_election.cc:288] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [CANDIDATE]: Term 1 election: Requested vote from peers 0dcfedacdf044397a1333c111dca7ba4 (127.29.142.130:38157), bde0c6f198b54b6a96850cc62e3a53e3 (127.29.142.131:42607)
I20260806 06:39:47.726321 24576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3a0245b4c8046098a4fdeedb0d36f76" candidate_uuid: "bb017e5573ba4713859fe9d6119b31ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dcfedacdf044397a1333c111dca7ba4"
I20260806 06:39:47.726361 24744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3a0245b4c8046098a4fdeedb0d36f76" candidate_uuid: "bb017e5573ba4713859fe9d6119b31ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bde0c6f198b54b6a96850cc62e3a53e3"
I20260806 06:39:47.726435 24744 raft_consensus.cc:3037] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:47.726433 24576 raft_consensus.cc:3037] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:47.727488 24576 raft_consensus.cc:2454] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb017e5573ba4713859fe9d6119b31ac in term 1.
I20260806 06:39:47.727488 24744 raft_consensus.cc:2454] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb017e5573ba4713859fe9d6119b31ac in term 1.
I20260806 06:39:47.727716 24312 leader_election.cc:302] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dcfedacdf044397a1333c111dca7ba4, bb017e5573ba4713859fe9d6119b31ac; no voters:
I20260806 06:39:47.727821 24835 raft_consensus.cc:2781] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:47.728020 24835 raft_consensus.cc:686] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [term 1 LEADER]: Becoming Leader. State: Replica: bb017e5573ba4713859fe9d6119b31ac, State: Running, Role: LEADER
I20260806 06:39:47.728143 24835 consensus_queue.cc:237] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb017e5573ba4713859fe9d6119b31ac" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38959 } } peers { permanent_uuid: "0dcfedacdf044397a1333c111dca7ba4" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38157 } } peers { permanent_uuid: "bde0c6f198b54b6a96850cc62e3a53e3" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 42607 } }
I20260806 06:39:47.729229 24200 catalog_manager.cc:5495] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac reported cstate change: term changed from 0 to 1, leader changed from <none> to bb017e5573ba4713859fe9d6119b31ac (127.29.142.129). New cstate: current_term: 1 leader_uuid: "bb017e5573ba4713859fe9d6119b31ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb017e5573ba4713859fe9d6119b31ac" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 38959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0dcfedacdf044397a1333c111dca7ba4" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bde0c6f198b54b6a96850cc62e3a53e3" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 42607 } health_report { overall_health: UNKNOWN } } }
I20260806 06:39:47.742139 24817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.739595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44352 (local address 127.29.142.129:38959)
0806 06:39:47.739681 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:47.739684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:47.739694 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:39:47.739740 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:47.739741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:47.739773 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:47.739864 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:39:47.740031 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.740523 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.741318 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:47.741532 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:47.741611 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:47.741769 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:39:47.741875 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:39:47.741990 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:47.742023 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 06:39:47.742035 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 06:39:47.744465 24744 raft_consensus.cc:1261] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3 [term 1 FOLLOWER]: Refusing update from remote peer bb017e5573ba4713859fe9d6119b31ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:47.744503 24576 raft_consensus.cc:1261] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4 [term 1 FOLLOWER]: Refusing update from remote peer bb017e5573ba4713859fe9d6119b31ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:47.744905 24835 consensus_queue.cc:1035] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dcfedacdf044397a1333c111dca7ba4" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 38157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:47.745062 24838 consensus_queue.cc:1035] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "bde0c6f198b54b6a96850cc62e3a53e3" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 42607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:47.747977 24844 mvcc.cc:205] Tried to move back new op lower bound from 7315449396197736448 to 7315449396135366656. Current Snapshot: MvccSnapshot[applied={T|T < 7315449396197736448}]
I20260806 06:39:47.748474 24846 mvcc.cc:205] Tried to move back new op lower bound from 7315449396197736448 to 7315449396135366656. Current Snapshot: MvccSnapshot[applied={T|T < 7315449396197736448}]
I20260806 06:39:47.749014 24845 mvcc.cc:205] Tried to move back new op lower bound from 7315449396197736448 to 7315449396135366656. Current Snapshot: MvccSnapshot[applied={T|T < 7315449396197736448}]
I20260806 06:39:47.751076 24200 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:47.751178 24200 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:47.752053 24200 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42c4d440bcc04fe5a9d92844658d7929: Sending DeleteTablet for 3 replicas of tablet c3a0245b4c8046098a4fdeedb0d36f76
I20260806 06:39:47.752460 24550 tablet_service.cc:1514] Processing DeleteTablet for tablet c3a0245b4c8046098a4fdeedb0d36f76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:47 UTC) from {username='slave'} at 127.0.0.1:48440
I20260806 06:39:47.752457 24373 tablet_service.cc:1514] Processing DeleteTablet for tablet c3a0245b4c8046098a4fdeedb0d36f76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:47 UTC) from {username='slave'} at 127.0.0.1:44348
I20260806 06:39:47.752697 24856 tablet_replica.cc:330] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4: stopping tablet replica
I20260806 06:39:47.752708 24718 tablet_service.cc:1514] Processing DeleteTablet for tablet c3a0245b4c8046098a4fdeedb0d36f76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:47 UTC) from {username='slave'} at 127.0.0.1:47450
I20260806 06:39:47.752782 24856 raft_consensus.cc:2229] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:39:47.752869 24856 raft_consensus.cc:2258] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:47.752992 24858 tablet_replica.cc:330] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3: stopping tablet replica
I20260806 06:39:47.753103 24858 raft_consensus.cc:2229] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:39:47.753211 24858 raft_consensus.cc:2258] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:47.753369 24856 ts_tablet_manager.cc:1905] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:47.753413 24857 tablet_replica.cc:330] T c3a0245b4c8046098a4fdeedb0d36f76 P bb017e5573ba4713859fe9d6119b31ac: stopping tablet replica
I20260806 06:39:47.753602 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 24287
I20260806 06:39:47.763497 24856 ts_tablet_manager.cc:1918] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:47.763602 24856 log.cc:1196] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/c3a0245b4c8046098a4fdeedb0d36f76
I20260806 06:39:47.763849 24856 ts_tablet_manager.cc:1939] T c3a0245b4c8046098a4fdeedb0d36f76 P 0dcfedacdf044397a1333c111dca7ba4: Deleting consensus metadata
I20260806 06:39:47.765060 24184 catalog_manager.cc:4841] TS 0dcfedacdf044397a1333c111dca7ba4 (127.29.142.130:38157): tablet c3a0245b4c8046098a4fdeedb0d36f76 (table test-table [id=88cd5bef576846ada8dc94211ad16d56]) successfully deleted
I20260806 06:39:47.769306 24858 ts_tablet_manager.cc:1905] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:47.771718 24858 ts_tablet_manager.cc:1918] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:47.771843 24858 log.cc:1196] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/c3a0245b4c8046098a4fdeedb0d36f76
I20260806 06:39:47.772140 24858 ts_tablet_manager.cc:1939] T c3a0245b4c8046098a4fdeedb0d36f76 P bde0c6f198b54b6a96850cc62e3a53e3: Deleting consensus metadata
I20260806 06:39:47.772778 24183 catalog_manager.cc:4841] TS bde0c6f198b54b6a96850cc62e3a53e3 (127.29.142.131:42607): tablet c3a0245b4c8046098a4fdeedb0d36f76 (table test-table [id=88cd5bef576846ada8dc94211ad16d56]) successfully deleted
W20260806 06:39:47.774346 24182 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:38959 (error 108)
I20260806 06:39:47.774610 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 24473
I20260806 06:39:47.778337 24814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:47.774821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:38959 (local address 127.0.0.1:44354)
0806 06:39:47.778226 (+ 3405us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:47.778288 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3370}
W20260806 06:39:47.778478 24182 catalog_manager.cc:4584] TS bb017e5573ba4713859fe9d6119b31ac (127.29.142.129:38959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c3a0245b4c8046098a4fdeedb0d36f76: Network error: Client connection negotiation failed: client connection to 127.29.142.129:38959: connect: Connection refused (error 111)
I20260806 06:39:47.789233 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 24640
I20260806 06:39:47.807400 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 24159
2026-08-06T06:39:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (818 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:39:47 dist-test-slave-3kk6 krb5kdc[24864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:39:47 dist-test-slave-3kk6 krb5kdc[24864](info): set up 2 sockets
Aug 06 06:39:47 dist-test-slave-3kk6 krb5kdc[24864](info): commencing operation
krb5kdc: starting...
W20260806 06:39:49.057678 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:39:49 dist-test-slave-3kk6 krb5kdc[24864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:39:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:39:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:39:49.187594 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:35215
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:44729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:35215
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:49.276942 24879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:49.277221 24879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:49.277267 24879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:49.278491 24879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:49.278573 24879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:49.278599 24879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:49.278622 24879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:49.278700 24879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:49.280370 24879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:44729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:35215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:35215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:49.280694 24879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:49.280964 24879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:49.286536 24891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:49.286614 24893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:49.286573 24889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:49.286787 24879 server_base.cc:1034] running on GCE node
I20260806 06:39:49.287067 24879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:49.287417 24879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:49.288612 24879 hybrid_clock.cc:648] HybridClock initialized: now 1785998389288599 us; error 73 us; skew 500 ppm
Aug 06 06:39:49 dist-test-slave-3kk6 krb5kdc[24864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998389, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:49.295948 24879 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:39:49.296526 24879 webserver.cc:460] Webserver started at http://127.29.142.190:36399/ using document root <none> and password file <none>
I20260806 06:39:49.296772 24879 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:49.296819 24879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:49.296924 24879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:49.298053 24879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "089d70b02add4403a3cd9702b3baeed6"
format_stamp: "Formatted at 2026-08-06 06:39:49 on dist-test-slave-3kk6"
server_key: "10ace6a0ea8e0dfbcda9d7f3ffbca476"
server_key_iv: "121a016057424d4f5814f8f9b23ce9c8"
server_key_version: "encryptionkey@0"
I20260806 06:39:49.298408 24879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "089d70b02add4403a3cd9702b3baeed6"
format_stamp: "Formatted at 2026-08-06 06:39:49 on dist-test-slave-3kk6"
server_key: "10ace6a0ea8e0dfbcda9d7f3ffbca476"
server_key_iv: "121a016057424d4f5814f8f9b23ce9c8"
server_key_version: "encryptionkey@0"
I20260806 06:39:49.299968 24879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:39:49.300853 24903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:49.301071 24879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:49.301179 24879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "089d70b02add4403a3cd9702b3baeed6"
format_stamp: "Formatted at 2026-08-06 06:39:49 on dist-test-slave-3kk6"
server_key: "10ace6a0ea8e0dfbcda9d7f3ffbca476"
server_key_iv: "121a016057424d4f5814f8f9b23ce9c8"
server_key_version: "encryptionkey@0"
I20260806 06:39:49.301260 24879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:49.310206 24879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:49.312716 24879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:49.312860 24879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:49.317366 24879 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:35215
I20260806 06:39:49.317449 24971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:35215 every 8 connection(s)
I20260806 06:39:49.317835 24879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:49.319013 24972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:49.321573 24972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6: Bootstrap starting.
I20260806 06:39:49.322254 24972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:49.322576 24972 log.cc:824] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:49.323431 24972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6: No bootstrap required, opened a new log
I20260806 06:39:49.323818 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 24879
I20260806 06:39:49.323933 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:39:49.324148 30266 external_mini_cluster.cc:1428] Setting key 3a86cc8ac0a427d1e783fdd9d5968e5c
I20260806 06:39:49.326499 24972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089d70b02add4403a3cd9702b3baeed6" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35215 } }
I20260806 06:39:49.326725 24972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:49.326766 24972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089d70b02add4403a3cd9702b3baeed6, State: Initialized, Role: FOLLOWER
I20260806 06:39:49.326901 24972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089d70b02add4403a3cd9702b3baeed6" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35215 } }
I20260806 06:39:49.326992 24972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:49.327025 24972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:49.327073 24972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:49.327870 24972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089d70b02add4403a3cd9702b3baeed6" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35215 } }
I20260806 06:39:49.327998 24972 leader_election.cc:302] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 089d70b02add4403a3cd9702b3baeed6; no voters:
I20260806 06:39:49.328233 24972 leader_election.cc:288] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:39:49.328256 24977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:49.328385 24977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [term 1 LEADER]: Becoming Leader. State: Replica: 089d70b02add4403a3cd9702b3baeed6, State: Running, Role: LEADER
I20260806 06:39:49.328533 24972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:49.328507 24977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089d70b02add4403a3cd9702b3baeed6" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35215 } }
I20260806 06:39:49.328879 24977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 089d70b02add4403a3cd9702b3baeed6. Latest consensus state: current_term: 1 leader_uuid: "089d70b02add4403a3cd9702b3baeed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089d70b02add4403a3cd9702b3baeed6" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35215 } } }
I20260806 06:39:49.328975 24977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:49.328891 24979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "089d70b02add4403a3cd9702b3baeed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089d70b02add4403a3cd9702b3baeed6" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 35215 } } }
I20260806 06:39:49.329152 24979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:49.329212 24984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:49.330078 24984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:39:49.332217 24984 catalog_manager.cc:1347] Generated new cluster ID: b2ab26e392da45b3b628560f407f82bf
I20260806 06:39:49.332284 24984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:39:49 dist-test-slave-3kk6 krb5kdc[24864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:49.337507 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.324774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.29.142.190:35215)
0806 06:39:49.325011 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:49.325022 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:49.325064 (+ 42us) server_negotiation.cc:407] Connection header received
0806 06:39:49.325522 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:49.325533 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:49.325601 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:49.325782 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:49.326447 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.327882 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.328970 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.329609 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.329692 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.333313 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:49.333317 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:49.333317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:49.333330 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:49.335505 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:49.335969 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:49.335972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:49.335973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:49.336079 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:49.336327 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:49.336328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:49.336328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:49.336634 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:49.336810 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:49.336946 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 06:39:49.337086 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":109,"threads_started":1}
I20260806 06:39:49.350548 24984 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:39:49.351394 24984 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:49.360256 24984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6: Generated new TSK 0
I20260806 06:39:49.360522 24984 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.142.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
Entry for principal kudu/127.29.142.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab.
I20260806 06:39:49.394282 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35215
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:44729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:49.481791 25008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:49.482041 25008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:49.482067 25008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:49.483457 25008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:49.483508 25008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:49.483537 25008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:39:49.485054 25008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:44729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.129.keytab
--principal=kudu/127.29.142.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:49.485309 25008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:49.485575 25008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:49.491015 25016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:49.491077 25015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:49.491158 25008 server_base.cc:1034] running on GCE node
W20260806 06:39:49.491298 25018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:49.491504 25008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:49.491786 25008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:49.492987 25008 hybrid_clock.cc:648] HybridClock initialized: now 1785998389492989 us; error 48 us; skew 500 ppm
Aug 06 06:39:49 dist-test-slave-3kk6 krb5kdc[24864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998389, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:49.495839 25008 init.cc:377] Logged in from keytab as kudu/127.29.142.129@KRBTEST.COM (short username kudu)
I20260806 06:39:49.496287 25008 webserver.cc:460] Webserver started at http://127.29.142.129:42861/ using document root <none> and password file <none>
I20260806 06:39:49.496541 25008 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:49.496590 25008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:49.496747 25008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:49.497900 25008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "723f58081e974f7087d06f9599769747"
format_stamp: "Formatted at 2026-08-06 06:39:49 on dist-test-slave-3kk6"
server_key: "4ff5c674ff606c2c946ca806bb02a3c8"
server_key_iv: "281fc9d3dc38d290281a8bdb37ed265e"
server_key_version: "encryptionkey@0"
I20260806 06:39:49.498297 25008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "723f58081e974f7087d06f9599769747"
format_stamp: "Formatted at 2026-08-06 06:39:49 on dist-test-slave-3kk6"
server_key: "4ff5c674ff606c2c946ca806bb02a3c8"
server_key_iv: "281fc9d3dc38d290281a8bdb37ed265e"
server_key_version: "encryptionkey@0"
I20260806 06:39:49.499976 25008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:39:49.500861 25026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:49.501070 25008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:39:49.501170 25008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "723f58081e974f7087d06f9599769747"
format_stamp: "Formatted at 2026-08-06 06:39:49 on dist-test-slave-3kk6"
server_key: "4ff5c674ff606c2c946ca806bb02a3c8"
server_key_iv: "281fc9d3dc38d290281a8bdb37ed265e"
server_key_version: "encryptionkey@0"
I20260806 06:39:49.501237 25008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:49.520138 25008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:49.522644 25008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:49.522882 25008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:49.523119 25008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:49.523483 25008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:49.523521 25008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:49.523545 25008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:49.523594 25008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:49.529850 25008 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:44801
I20260806 06:39:49.529955 25175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:44801 every 8 connection(s)
I20260806 06:39:49.530345 25008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:39:49.530541 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 25008
I20260806 06:39:49.530726 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:39:49.530987 30266 external_mini_cluster.cc:1428] Setting key 65dfec5ed54a4606be46822c912889e2
Aug 06 06:39:49 dist-test-slave-3kk6 krb5kdc[24864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998389, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.129@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.142.130@KRBTEST.COM; defaulting to no policy
I20260806 06:39:49.541610 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.531357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:38415 (local address 127.29.142.190:35215)
0806 06:39:49.531410 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:49.531412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:49.531708 (+ 296us) server_negotiation.cc:407] Connection header received
0806 06:39:49.532265 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:49.532266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:49.532279 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:49.532324 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:49.533500 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.534014 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.535306 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.535650 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.535737 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.538055 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:49.538058 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:49.538058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:49.538069 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:49.540090 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:49.540565 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:49.540567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:49.540568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:49.540643 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:49.540923 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:49.540925 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:49.540925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:49.541097 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:49.541179 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:49.541451 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 06:39:49.541535 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:39:49.541790 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.531340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:35215 (local address 127.29.142.129:38415)
0806 06:39:49.531534 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:49.531583 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:49.532147 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:49.532429 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:49.532435 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:49.533054 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:49.533370 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:49.533378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.534124 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.534125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:49.535206 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:49.535209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.535750 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.535751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:49.535860 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.536470 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:49.536480 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:49.537947 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:49.540206 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:49.540215 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:49.540217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:49.540478 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:49.540720 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:49.540721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:49.540721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:49.540845 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:49.541163 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:49.541169 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:49.541314 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 06:39:49.541410 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:39:49.541552 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260806 06:39:49.542701 25176 heartbeater.cc:347] Connected to a master server at 127.29.142.190:35215
I20260806 06:39:49.542842 25176 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:49.543057 25176 heartbeater.cc:511] Master 127.29.142.190:35215 requested a full tablet report, sending...
I20260806 06:39:49.543792 24922 ts_manager.cc:194] Registered new tserver with Master: 723f58081e974f7087d06f9599769747 (127.29.142.129:44801)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.130@KRBTEST.COM" created.
I20260806 06:39:49.544492 24922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.129@KRBTEST.COM'} at 127.29.142.129:38415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
Entry for principal kudu/127.29.142.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab.
I20260806 06:39:49.561312 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35215
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:44729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:49.654049 25184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:49.654313 25184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:49.654341 25184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:49.655692 25184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:49.655766 25184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:49.655797 25184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:39:49.657505 25184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:44729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.130.keytab
--principal=kudu/127.29.142.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:49.657804 25184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:49.658067 25184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:49.663697 25195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:49.663702 25197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:49.663733 25199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:49.663991 25184 server_base.cc:1034] running on GCE node
I20260806 06:39:49.664234 25184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:49.664511 25184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:49.665720 25184 hybrid_clock.cc:648] HybridClock initialized: now 1785998389665701 us; error 30 us; skew 500 ppm
Aug 06 06:39:49 dist-test-slave-3kk6 krb5kdc[24864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998389, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:49.668733 25184 init.cc:377] Logged in from keytab as kudu/127.29.142.130@KRBTEST.COM (short username kudu)
I20260806 06:39:49.669204 25184 webserver.cc:460] Webserver started at http://127.29.142.130:42219/ using document root <none> and password file <none>
I20260806 06:39:49.669425 25184 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:49.669472 25184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:49.669574 25184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:49.670710 25184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "1afb022979064a5d89a15f25298c1740"
format_stamp: "Formatted at 2026-08-06 06:39:49 on dist-test-slave-3kk6"
server_key: "438ff1681801a0f4f29911d88fbda392"
server_key_iv: "bab5fe8b764d650f5e23b9c75eb941bc"
server_key_version: "encryptionkey@0"
I20260806 06:39:49.671097 25184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "1afb022979064a5d89a15f25298c1740"
format_stamp: "Formatted at 2026-08-06 06:39:49 on dist-test-slave-3kk6"
server_key: "438ff1681801a0f4f29911d88fbda392"
server_key_iv: "bab5fe8b764d650f5e23b9c75eb941bc"
server_key_version: "encryptionkey@0"
I20260806 06:39:49.672619 25184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:39:49.673482 25206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:49.673697 25184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:49.673818 25184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "1afb022979064a5d89a15f25298c1740"
format_stamp: "Formatted at 2026-08-06 06:39:49 on dist-test-slave-3kk6"
server_key: "438ff1681801a0f4f29911d88fbda392"
server_key_iv: "bab5fe8b764d650f5e23b9c75eb941bc"
server_key_version: "encryptionkey@0"
I20260806 06:39:49.673887 25184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:49.692633 25184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:49.695173 25184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:49.695322 25184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:49.695544 25184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:49.695907 25184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:49.695945 25184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:49.695969 25184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:49.696018 25184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:49.702237 25184 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:41277
I20260806 06:39:49.702347 25356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:41277 every 8 connection(s)
I20260806 06:39:49.702744 25184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:39:49.708027 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 25184
I20260806 06:39:49.708174 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:39:49.708398 30266 external_mini_cluster.cc:1428] Setting key 69a5db42322b8aded8b33bf2a59789b8
Aug 06 06:39:49 dist-test-slave-3kk6 krb5kdc[24864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998389, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.130@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:49.713621 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.703595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:43621 (local address 127.29.142.190:35215)
0806 06:39:49.703657 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:49.703659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:49.703902 (+ 243us) server_negotiation.cc:407] Connection header received
0806 06:39:49.704538 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:49.704539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:49.704551 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:49.704623 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:49.705664 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.706213 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.707291 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.707487 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.707558 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.710358 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:49.710360 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:49.710360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:49.710370 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:49.712062 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:49.712534 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:49.712536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:49.712536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:49.712606 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:49.712883 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:49.712884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:49.712884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:49.713034 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:49.713118 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:49.713428 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 06:39:49.713522 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:39:49.713829 25359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.703536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:35215 (local address 127.29.142.130:43621)
0806 06:39:49.703718 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:49.703769 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:49.704416 (+ 647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:49.704710 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:49.704714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:49.705353 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:49.705567 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:49.705572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.706321 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.706322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:49.707189 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:49.707192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.707589 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.707590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:49.707702 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.708531 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:49.708540 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:49.710261 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:49.712164 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:49.712173 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:49.712175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:49.712433 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:49.712680 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:49.712680 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:49.712681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:49.712805 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:49.713107 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:49.713115 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:49.713274 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 06:39:49.713389 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 06:39:49.713566 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260806 06:39:49.714403 25357 heartbeater.cc:347] Connected to a master server at 127.29.142.190:35215
I20260806 06:39:49.714529 25357 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:49.714788 25357 heartbeater.cc:511] Master 127.29.142.190:35215 requested a full tablet report, sending...
I20260806 06:39:49.715286 24922 ts_manager.cc:194] Registered new tserver with Master: 1afb022979064a5d89a15f25298c1740 (127.29.142.130:41277)
I20260806 06:39:49.715682 24922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.130@KRBTEST.COM'} at 127.29.142.130:43621
WARNING: no policy specified for kudu/127.29.142.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
Entry for principal kudu/127.29.142.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab.
I20260806 06:39:49.738807 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35215
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:44729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:49.825975 25364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:49.826215 25364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:49.826241 25364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:49.827687 25364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:49.827744 25364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:49.827775 25364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:39:49.829736 25364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:44729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.131.keytab
--principal=kudu/127.29.142.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:35215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:49.830021 25364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:49.830281 25364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:39:49.835920 25364 server_base.cc:1034] running on GCE node
W20260806 06:39:49.836030 25375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:49.836030 25379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:49.836251 25373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:49.836477 25364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:49.836788 25364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:49.837960 25364 hybrid_clock.cc:648] HybridClock initialized: now 1785998389837939 us; error 46 us; skew 500 ppm
Aug 06 06:39:49 dist-test-slave-3kk6 krb5kdc[24864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998389, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:49.840644 25364 init.cc:377] Logged in from keytab as kudu/127.29.142.131@KRBTEST.COM (short username kudu)
I20260806 06:39:49.841132 25364 webserver.cc:460] Webserver started at http://127.29.142.131:40223/ using document root <none> and password file <none>
I20260806 06:39:49.841384 25364 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:49.841457 25364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:49.841614 25364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:49.842787 25364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "37178320714e4005bfec37c1b6254568"
format_stamp: "Formatted at 2026-08-06 06:39:49 on dist-test-slave-3kk6"
server_key: "d79c8e6184622238df06e8bec57ae98b"
server_key_iv: "eecf052b19b34f94a20643da289435ad"
server_key_version: "encryptionkey@0"
I20260806 06:39:49.843183 25364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "37178320714e4005bfec37c1b6254568"
format_stamp: "Formatted at 2026-08-06 06:39:49 on dist-test-slave-3kk6"
server_key: "d79c8e6184622238df06e8bec57ae98b"
server_key_iv: "eecf052b19b34f94a20643da289435ad"
server_key_version: "encryptionkey@0"
I20260806 06:39:49.844584 25364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:39:49.845355 25387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:49.845535 25364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:39:49.845649 25364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "37178320714e4005bfec37c1b6254568"
format_stamp: "Formatted at 2026-08-06 06:39:49 on dist-test-slave-3kk6"
server_key: "d79c8e6184622238df06e8bec57ae98b"
server_key_iv: "eecf052b19b34f94a20643da289435ad"
server_key_version: "encryptionkey@0"
I20260806 06:39:49.845727 25364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:49.865303 25364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:49.867591 25364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:49.867784 25364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:49.868049 25364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:49.868415 25364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:49.868474 25364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:49.868523 25364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:49.868548 25364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:49.874914 25364 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:35081
I20260806 06:39:49.874964 25544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:35081 every 8 connection(s)
I20260806 06:39:49.875347 25364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:39:49 dist-test-slave-3kk6 krb5kdc[24864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998389, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.131@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:39:49.885151 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 25364
I20260806 06:39:49.885226 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.876249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:40701 (local address 127.29.142.190:35215)
0806 06:39:49.876331 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:49.876334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:49.876531 (+ 197us) server_negotiation.cc:407] Connection header received
0806 06:39:49.877145 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:49.877146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:49.877156 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:49.877214 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:49.878315 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.878902 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.879930 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.880119 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.880180 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.882550 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:49.882551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:49.882551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:49.882558 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:49.883911 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:49.884322 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:49.884323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:49.884324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:49.884377 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:49.884598 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:49.884598 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:49.884598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:49.884701 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:49.884752 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:49.885026 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 06:39:49.885110 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:39:49.885262 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:39:49.885367 25548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.876204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:35215 (local address 127.29.142.131:40701)
0806 06:39:49.876410 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:49.876451 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:49.877049 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:49.877303 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:49.877307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:49.878003 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:49.878219 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:49.878224 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.878995 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.878996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:49.879838 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:49.879841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.880239 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.880240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:49.880348 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.880994 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:39:49.881004 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:49.882463 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:49.883994 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:49.884001 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:49.884003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:49.884253 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:49.884438 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:39:49.884438 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:39:49.884439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:39:49.884543 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:39:49.884759 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:49.884763 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:49.884880 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 06:39:49.884980 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:39:49.885110 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":117,"threads_started":1}
I20260806 06:39:49.885507 30266 external_mini_cluster.cc:1428] Setting key fdb6a44bae480812f52cc294ef50c3a1
I20260806 06:39:49.886065 25545 heartbeater.cc:347] Connected to a master server at 127.29.142.190:35215
I20260806 06:39:49.886189 25545 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:49.886412 25545 heartbeater.cc:511] Master 127.29.142.190:35215 requested a full tablet report, sending...
I20260806 06:39:49.886906 24921 ts_manager.cc:194] Registered new tserver with Master: 37178320714e4005bfec37c1b6254568 (127.29.142.131:35081)
I20260806 06:39:49.887307 24921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.142.131@KRBTEST.COM'} at 127.29.142.131:40701
I20260806 06:39:49.897785 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:39:49.906039 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.899413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43860 (local address 127.29.142.190:35215)
0806 06:39:49.899474 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:49.899476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:49.899619 (+ 143us) server_negotiation.cc:407] Connection header received
0806 06:39:49.899664 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:49.899665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:49.899675 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:49.899734 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:49.900334 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.900861 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.901614 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.901804 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.901858 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.903327 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:49.903328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:49.903328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:49.903335 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:49.904799 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:49.905259 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:49.905260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:49.905260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:49.905343 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:49.905541 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:49.905541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:49.905542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:49.905694 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:49.905775 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:49.905894 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 06:39:49.905966 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:39:49.907887 24921 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:39:49.909473 24921 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:39:49.916442 25564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.912569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52868 (local address 127.29.142.130:41277)
0806 06:39:49.912847 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:49.912853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:49.912863 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:39:49.912902 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:49.912904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:49.912922 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:49.913025 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:49.913167 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.913875 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.915251 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.915982 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.916075 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.916140 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:49.916308 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 06:39:49.916352 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":104,"threads_started":1}
I20260806 06:39:49.916440 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.912271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:41277 (local address 127.0.0.1:52868)
0806 06:39:49.912692 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:49.912701 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:49.912738 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:49.913023 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:49.913024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:49.913026 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:49.913086 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:49.913088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.913981 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.913982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:49.915152 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:49.915155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.916075 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.916076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:49.916176 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.916192 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 06:39:49.916290 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 06:39:49.916349 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":101,"threads_started":1}
I20260806 06:39:49.916615 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.912267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:44801 (local address 127.0.0.1:36260)
0806 06:39:49.912462 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:49.912511 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:49.912580 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:49.912985 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:49.912988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:49.912991 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:49.913080 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:49.913082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.914481 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.914482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:49.915500 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:49.915502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.916351 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.916352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:49.916414 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.916419 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:39:49.916479 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:39:49.916530 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":91,"thread_start_us":105,"threads_started":1}
I20260806 06:39:49.917026 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.912268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:35081 (local address 127.0.0.1:49918)
0806 06:39:49.912571 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:49.912585 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:49.912646 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:49.913203 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:49.913204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:49.913206 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:49.913245 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:49.913246 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.914532 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.914533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:49.915735 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:49.915739 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.916781 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.916782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:49.916862 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.916866 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:39:49.916921 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:39:49.916962 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":72,"threads_started":1}
I20260806 06:39:49.917179 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.912376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36260 (local address 127.29.142.129:44801)
0806 06:39:49.912806 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:49.912810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:49.912820 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:39:49.912856 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:49.912858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:49.912875 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:49.913168 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:49.913250 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.914366 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.915585 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.916244 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.916962 (+ 718us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.917024 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:49.917052 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 06:39:49.917104 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":104,"threads_started":1}
I20260806 06:39:49.917294 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:49.912528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49918 (local address 127.29.142.131:35081)
0806 06:39:49.912737 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:49.912743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:49.912755 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:39:49.912824 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:49.912826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:49.912844 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:49.912927 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:49.913370 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.914433 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.915834 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:49.916608 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:49.916683 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:49.916741 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:49.917141 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 06:39:49.917188 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":89,"threads_started":1}
I20260806 06:39:49.917296 25270 tablet_service.cc:1467] Processing CreateTablet for tablet 60c22dbd53fb4e009ace8582dfac697d (DEFAULT_TABLE table=test-table [id=edad23bd7672410fb0ad6b78aab6c9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:49.917718 25270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60c22dbd53fb4e009ace8582dfac697d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:49.917907 25088 tablet_service.cc:1467] Processing CreateTablet for tablet 60c22dbd53fb4e009ace8582dfac697d (DEFAULT_TABLE table=test-table [id=edad23bd7672410fb0ad6b78aab6c9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:49.918238 25088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60c22dbd53fb4e009ace8582dfac697d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:49.920645 25565 tablet_bootstrap.cc:492] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740: Bootstrap starting.
I20260806 06:39:49.920877 25566 tablet_bootstrap.cc:492] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747: Bootstrap starting.
I20260806 06:39:49.921484 25444 tablet_service.cc:1467] Processing CreateTablet for tablet 60c22dbd53fb4e009ace8582dfac697d (DEFAULT_TABLE table=test-table [id=edad23bd7672410fb0ad6b78aab6c9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:49.921648 25566 tablet_bootstrap.cc:654] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:49.921774 25565 tablet_bootstrap.cc:654] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:49.921865 25444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60c22dbd53fb4e009ace8582dfac697d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:49.922256 25566 log.cc:824] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:49.922353 25565 log.cc:824] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:49.923202 25565 tablet_bootstrap.cc:492] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740: No bootstrap required, opened a new log
I20260806 06:39:49.923202 25566 tablet_bootstrap.cc:492] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747: No bootstrap required, opened a new log
I20260806 06:39:49.923282 25566 ts_tablet_manager.cc:1397] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:39:49.923364 25565 ts_tablet_manager.cc:1397] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:39:49.924888 25570 tablet_bootstrap.cc:492] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568: Bootstrap starting.
I20260806 06:39:49.925539 25570 tablet_bootstrap.cc:654] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:49.925411 25566 raft_consensus.cc:348] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "723f58081e974f7087d06f9599769747" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44801 } } peers { permanent_uuid: "37178320714e4005bfec37c1b6254568" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35081 } } peers { permanent_uuid: "1afb022979064a5d89a15f25298c1740" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41277 } }
I20260806 06:39:49.925608 25566 raft_consensus.cc:374] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:49.925673 25566 raft_consensus.cc:729] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 723f58081e974f7087d06f9599769747, State: Initialized, Role: FOLLOWER
I20260806 06:39:49.925801 25566 consensus_queue.cc:260] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "723f58081e974f7087d06f9599769747" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44801 } } peers { permanent_uuid: "37178320714e4005bfec37c1b6254568" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35081 } } peers { permanent_uuid: "1afb022979064a5d89a15f25298c1740" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41277 } }
I20260806 06:39:49.925856 25565 raft_consensus.cc:348] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "723f58081e974f7087d06f9599769747" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44801 } } peers { permanent_uuid: "37178320714e4005bfec37c1b6254568" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35081 } } peers { permanent_uuid: "1afb022979064a5d89a15f25298c1740" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41277 } }
I20260806 06:39:49.926038 25565 raft_consensus.cc:374] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:49.926088 25565 raft_consensus.cc:729] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1afb022979064a5d89a15f25298c1740, State: Initialized, Role: FOLLOWER
I20260806 06:39:49.926127 25566 ts_tablet_manager.cc:1428] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:39:49.926180 25570 log.cc:824] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:49.926276 25176 heartbeater.cc:503] Master 127.29.142.190:35215 was elected leader, sending a full tablet report...
I20260806 06:39:49.926263 25565 consensus_queue.cc:260] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "723f58081e974f7087d06f9599769747" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44801 } } peers { permanent_uuid: "37178320714e4005bfec37c1b6254568" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35081 } } peers { permanent_uuid: "1afb022979064a5d89a15f25298c1740" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41277 } }
I20260806 06:39:49.926599 25565 ts_tablet_manager.cc:1428] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:39:49.926635 25357 heartbeater.cc:503] Master 127.29.142.190:35215 was elected leader, sending a full tablet report...
I20260806 06:39:49.927457 25570 tablet_bootstrap.cc:492] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568: No bootstrap required, opened a new log
I20260806 06:39:49.927528 25570 ts_tablet_manager.cc:1397] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:39:49.929152 25570 raft_consensus.cc:348] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "723f58081e974f7087d06f9599769747" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44801 } } peers { permanent_uuid: "37178320714e4005bfec37c1b6254568" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35081 } } peers { permanent_uuid: "1afb022979064a5d89a15f25298c1740" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41277 } }
I20260806 06:39:49.929309 25570 raft_consensus.cc:374] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:49.929356 25570 raft_consensus.cc:729] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37178320714e4005bfec37c1b6254568, State: Initialized, Role: FOLLOWER
I20260806 06:39:49.929468 25570 consensus_queue.cc:260] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "723f58081e974f7087d06f9599769747" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44801 } } peers { permanent_uuid: "37178320714e4005bfec37c1b6254568" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35081 } } peers { permanent_uuid: "1afb022979064a5d89a15f25298c1740" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41277 } }
I20260806 06:39:49.929788 25570 ts_tablet_manager.cc:1428] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:39:49.929855 25545 heartbeater.cc:503] Master 127.29.142.190:35215 was elected leader, sending a full tablet report...
W20260806 06:39:49.953384 25358 tablet.cc:2369] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:39:50.031576 25178 tablet.cc:2369] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:39:50.035827 25573 raft_consensus.cc:482] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:39:50.035982 25573 raft_consensus.cc:504] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "723f58081e974f7087d06f9599769747" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44801 } } peers { permanent_uuid: "37178320714e4005bfec37c1b6254568" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35081 } } peers { permanent_uuid: "1afb022979064a5d89a15f25298c1740" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41277 } }
I20260806 06:39:50.036393 25573 leader_election.cc:288] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 723f58081e974f7087d06f9599769747 (127.29.142.129:44801), 37178320714e4005bfec37c1b6254568 (127.29.142.131:35081)
I20260806 06:39:50.039505 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.036725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:58883 (local address 127.29.142.129:44801)
0806 06:39:50.036781 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:50.036784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:50.036796 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:39:50.036900 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:50.036902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:50.036922 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:50.036995 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:50.037179 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.037678 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.038717 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.039101 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.039209 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.039219 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:50.039383 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 06:39:50.039424 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:39:50.039505 25359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.036611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:44801 (local address 127.29.142.130:58883)
0806 06:39:50.036718 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:50.036734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:50.036796 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:50.036999 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:50.037000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:50.037002 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:50.037089 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.037091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.037772 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.037773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.038565 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.038567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.039209 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.039210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.039294 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.039297 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:39:50.039374 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:39:50.039422 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":3}
I20260806 06:39:50.039759 25575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.036624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:35081 (local address 127.29.142.130:58115)
0806 06:39:50.036877 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:50.036888 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:50.036980 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:50.037147 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:50.037148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:50.037150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:50.037248 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.037250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.037896 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.037898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.038743 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.038747 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.039330 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.039331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.039414 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.039423 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 06:39:50.039491 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:39:50.039532 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":35,"thread_start_us":156,"threads_started":1}
I20260806 06:39:50.039765 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.036771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:58115 (local address 127.29.142.131:35081)
0806 06:39:50.036858 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:50.036860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:50.036960 (+ 100us) server_negotiation.cc:407] Connection header received
0806 06:39:50.037051 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:50.037053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:50.037063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:50.037142 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:50.037338 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.037805 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.038847 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.039234 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.039327 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.039336 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:50.039497 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 06:39:50.039539 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 06:39:50.039969 25117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60c22dbd53fb4e009ace8582dfac697d" candidate_uuid: "1afb022979064a5d89a15f25298c1740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "723f58081e974f7087d06f9599769747" is_pre_election: true
I20260806 06:39:50.040207 25117 raft_consensus.cc:2454] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1afb022979064a5d89a15f25298c1740 in term 0.
I20260806 06:39:50.040180 25469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60c22dbd53fb4e009ace8582dfac697d" candidate_uuid: "1afb022979064a5d89a15f25298c1740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37178320714e4005bfec37c1b6254568" is_pre_election: true
I20260806 06:39:50.040352 25469 raft_consensus.cc:2454] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1afb022979064a5d89a15f25298c1740 in term 0.
I20260806 06:39:50.040504 25211 leader_election.cc:302] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1afb022979064a5d89a15f25298c1740, 723f58081e974f7087d06f9599769747; no voters:
I20260806 06:39:50.040705 25573 raft_consensus.cc:2781] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:39:50.040747 25573 raft_consensus.cc:482] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:39:50.040766 25573 raft_consensus.cc:3037] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:50.041550 25573 raft_consensus.cc:504] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "723f58081e974f7087d06f9599769747" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44801 } } peers { permanent_uuid: "37178320714e4005bfec37c1b6254568" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35081 } } peers { permanent_uuid: "1afb022979064a5d89a15f25298c1740" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41277 } }
I20260806 06:39:50.041720 25573 leader_election.cc:288] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [CANDIDATE]: Term 1 election: Requested vote from peers 723f58081e974f7087d06f9599769747 (127.29.142.129:44801), 37178320714e4005bfec37c1b6254568 (127.29.142.131:35081)
I20260806 06:39:50.041924 25469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60c22dbd53fb4e009ace8582dfac697d" candidate_uuid: "1afb022979064a5d89a15f25298c1740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37178320714e4005bfec37c1b6254568"
I20260806 06:39:50.041970 25117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60c22dbd53fb4e009ace8582dfac697d" candidate_uuid: "1afb022979064a5d89a15f25298c1740" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "723f58081e974f7087d06f9599769747"
I20260806 06:39:50.042019 25469 raft_consensus.cc:3037] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:50.042066 25117 raft_consensus.cc:3037] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:50.042879 25117 raft_consensus.cc:2454] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1afb022979064a5d89a15f25298c1740 in term 1.
I20260806 06:39:50.042886 25469 raft_consensus.cc:2454] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1afb022979064a5d89a15f25298c1740 in term 1.
I20260806 06:39:50.043074 25209 leader_election.cc:302] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1afb022979064a5d89a15f25298c1740, 37178320714e4005bfec37c1b6254568; no voters:
I20260806 06:39:50.043200 25573 raft_consensus.cc:2781] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:50.043424 25573 raft_consensus.cc:686] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 1 LEADER]: Becoming Leader. State: Replica: 1afb022979064a5d89a15f25298c1740, State: Running, Role: LEADER
I20260806 06:39:50.043558 25573 consensus_queue.cc:237] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "723f58081e974f7087d06f9599769747" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44801 } } peers { permanent_uuid: "37178320714e4005bfec37c1b6254568" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35081 } } peers { permanent_uuid: "1afb022979064a5d89a15f25298c1740" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41277 } }
I20260806 06:39:50.044528 24921 catalog_manager.cc:5495] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1afb022979064a5d89a15f25298c1740 (127.29.142.130). New cstate: current_term: 1 leader_uuid: "1afb022979064a5d89a15f25298c1740" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "723f58081e974f7087d06f9599769747" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37178320714e4005bfec37c1b6254568" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1afb022979064a5d89a15f25298c1740" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 41277 } health_report { overall_health: HEALTHY } } }
I20260806 06:39:50.067528 25564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.064999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52880 (local address 127.29.142.130:41277)
0806 06:39:50.065074 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:50.065076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:50.065086 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:39:50.065137 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:50.065138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:50.065158 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:50.065231 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:39:50.065390 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.065899 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.066749 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.066962 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.067023 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.067177 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:39:50.067290 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:39:50.067356 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:50.067392 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 06:39:50.067412 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:39:50.069761 25117 raft_consensus.cc:1261] T 60c22dbd53fb4e009ace8582dfac697d P 723f58081e974f7087d06f9599769747 [term 1 FOLLOWER]: Refusing update from remote peer 1afb022979064a5d89a15f25298c1740: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:50.069748 25469 raft_consensus.cc:1261] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568 [term 1 FOLLOWER]: Refusing update from remote peer 1afb022979064a5d89a15f25298c1740: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:50.070200 25576 consensus_queue.cc:1035] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "723f58081e974f7087d06f9599769747" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 44801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:50.070354 25573 consensus_queue.cc:1035] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37178320714e4005bfec37c1b6254568" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:50.073220 25581 mvcc.cc:205] Tried to move back new op lower bound from 7315449405722480640 to 7315449405619122176. Current Snapshot: MvccSnapshot[applied={T|T < 7315449405722480640}]
I20260806 06:39:50.073432 25583 mvcc.cc:205] Tried to move back new op lower bound from 7315449405722480640 to 7315449405619122176. Current Snapshot: MvccSnapshot[applied={T|T < 7315449405722480640}]
I20260806 06:39:50.075588 24922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:50.075706 24922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:50.076645 24922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 089d70b02add4403a3cd9702b3baeed6: Sending DeleteTablet for 3 replicas of tablet 60c22dbd53fb4e009ace8582dfac697d
I20260806 06:39:50.077059 25088 tablet_service.cc:1514] Processing DeleteTablet for tablet 60c22dbd53fb4e009ace8582dfac697d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:50 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:36260
I20260806 06:39:50.077112 25444 tablet_service.cc:1514] Processing DeleteTablet for tablet 60c22dbd53fb4e009ace8582dfac697d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:50 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:49918
I20260806 06:39:50.077355 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 25008
I20260806 06:39:50.077378 25270 tablet_service.cc:1514] Processing DeleteTablet for tablet 60c22dbd53fb4e009ace8582dfac697d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:50 UTC) from {username='kudu', principal='kudu/127.29.142.190@KRBTEST.COM'} at 127.0.0.1:52868
I20260806 06:39:50.082521 25596 tablet_replica.cc:330] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740: stopping tablet replica
I20260806 06:39:50.082777 25596 raft_consensus.cc:2229] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:39:50.082986 25596 raft_consensus.cc:2258] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:50.083935 25596 ts_tablet_manager.cc:1905] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:50.084964 25595 tablet_replica.cc:330] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568: stopping tablet replica
I20260806 06:39:50.085116 25595 raft_consensus.cc:2229] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:39:50.085273 25595 raft_consensus.cc:2258] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:50.085836 25595 ts_tablet_manager.cc:1905] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:50.086038 25596 ts_tablet_manager.cc:1918] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:50.086118 25596 log.cc:1196] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/60c22dbd53fb4e009ace8582dfac697d
I20260806 06:39:50.086429 25596 ts_tablet_manager.cc:1939] T 60c22dbd53fb4e009ace8582dfac697d P 1afb022979064a5d89a15f25298c1740: Deleting consensus metadata
I20260806 06:39:50.087002 24905 catalog_manager.cc:4841] TS 1afb022979064a5d89a15f25298c1740 (127.29.142.130:41277): tablet 60c22dbd53fb4e009ace8582dfac697d (table test-table [id=edad23bd7672410fb0ad6b78aab6c9b1]) successfully deleted
I20260806 06:39:50.087456 25595 ts_tablet_manager.cc:1918] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:50.087522 25595 log.cc:1196] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/60c22dbd53fb4e009ace8582dfac697d
I20260806 06:39:50.087734 25595 ts_tablet_manager.cc:1939] T 60c22dbd53fb4e009ace8582dfac697d P 37178320714e4005bfec37c1b6254568: Deleting consensus metadata
I20260806 06:39:50.088157 24908 catalog_manager.cc:4841] TS 37178320714e4005bfec37c1b6254568 (127.29.142.131:35081): tablet 60c22dbd53fb4e009ace8582dfac697d (table test-table [id=edad23bd7672410fb0ad6b78aab6c9b1]) successfully deleted
W20260806 06:39:50.091264 24909 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:44801 (error 108)
I20260806 06:39:50.091413 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 25184
I20260806 06:39:50.094044 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.093860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:44801 (local address 127.0.0.1:36266)
0806 06:39:50.093936 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:50.093995 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:44801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 06:39:50.094208 24909 catalog_manager.cc:4584] TS 723f58081e974f7087d06f9599769747 (127.29.142.129:44801): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 60c22dbd53fb4e009ace8582dfac697d: Network error: Client connection negotiation failed: client connection to 127.29.142.129:44801: connect: Connection refused (error 111)
I20260806 06:39:50.105535 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 25364
I20260806 06:39:50.127749 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 24879
2026-08-06T06:39:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2322 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T06:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:39:50Z Disabled control of system clock
I20260806 06:39:50.165170 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:37067
--webserver_interface=127.29.142.190
--webserver_port=0
--builtin_ntp_servers=127.29.142.148:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:37067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:39:50.251331 25601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:50.251583 25601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:50.251605 25601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:50.252852 25601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:50.252907 25601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:50.252921 25601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:50.252934 25601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:50.252947 25601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:50.254370 25601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:37067
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:37067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:50.254660 25601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:50.255096 25601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:50.260681 25610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:50.260684 25608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:50.260684 25612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:50.260999 25601 server_base.cc:1034] running on GCE node
I20260806 06:39:50.261281 25601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:50.261652 25601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:50.262825 25601 hybrid_clock.cc:648] HybridClock initialized: now 1785998390262801 us; error 41 us; skew 500 ppm
I20260806 06:39:50.264257 25601 webserver.cc:460] Webserver started at http://127.29.142.190:39503/ using document root <none> and password file <none>
I20260806 06:39:50.264540 25601 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:50.264618 25601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:50.264777 25601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:50.265997 25601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "655961c6e79944d29d4b4eab2d86aa5d"
format_stamp: "Formatted at 2026-08-06 06:39:50 on dist-test-slave-3kk6"
server_key: "1a0a7d74eb6b0a8ff05bccedf5431d0d"
server_key_iv: "5da18a6450b2c79513540c22784512d7"
server_key_version: "encryptionkey@0"
I20260806 06:39:50.266409 25601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "655961c6e79944d29d4b4eab2d86aa5d"
format_stamp: "Formatted at 2026-08-06 06:39:50 on dist-test-slave-3kk6"
server_key: "1a0a7d74eb6b0a8ff05bccedf5431d0d"
server_key_iv: "5da18a6450b2c79513540c22784512d7"
server_key_version: "encryptionkey@0"
I20260806 06:39:50.267901 25601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:39:50.269027 25621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:50.269248 25601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:50.269347 25601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "655961c6e79944d29d4b4eab2d86aa5d"
format_stamp: "Formatted at 2026-08-06 06:39:50 on dist-test-slave-3kk6"
server_key: "1a0a7d74eb6b0a8ff05bccedf5431d0d"
server_key_iv: "5da18a6450b2c79513540c22784512d7"
server_key_version: "encryptionkey@0"
I20260806 06:39:50.269425 25601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:50.287791 25601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:50.288172 25601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:50.288342 25601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:50.292598 25601 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:37067
I20260806 06:39:50.292635 25696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:37067 every 8 connection(s)
I20260806 06:39:50.293087 25601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:50.294632 25697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:50.297672 25697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d: Bootstrap starting.
I20260806 06:39:50.298285 25697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:50.298543 25697 log.cc:824] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:50.299382 25697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d: No bootstrap required, opened a new log
I20260806 06:39:50.300669 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 25601
I20260806 06:39:50.300781 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:39:50.300990 30266 external_mini_cluster.cc:1428] Setting key 3020575ec14120a5da71e6c7df693727
I20260806 06:39:50.301167 25697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655961c6e79944d29d4b4eab2d86aa5d" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37067 } }
I20260806 06:39:50.301348 25697 raft_consensus.cc:374] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:50.301396 25697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655961c6e79944d29d4b4eab2d86aa5d, State: Initialized, Role: FOLLOWER
I20260806 06:39:50.301569 25697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655961c6e79944d29d4b4eab2d86aa5d" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37067 } }
I20260806 06:39:50.301677 25697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:50.301719 25697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:50.301762 25697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:50.302366 25697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655961c6e79944d29d4b4eab2d86aa5d" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37067 } }
I20260806 06:39:50.302497 25697 leader_election.cc:302] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 655961c6e79944d29d4b4eab2d86aa5d; no voters:
I20260806 06:39:50.302745 25697 leader_election.cc:288] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:39:50.302805 25702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:50.302978 25702 raft_consensus.cc:686] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [term 1 LEADER]: Becoming Leader. State: Replica: 655961c6e79944d29d4b4eab2d86aa5d, State: Running, Role: LEADER
I20260806 06:39:50.303066 25697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:50.303089 25702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655961c6e79944d29d4b4eab2d86aa5d" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37067 } }
I20260806 06:39:50.303522 25702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 655961c6e79944d29d4b4eab2d86aa5d. Latest consensus state: current_term: 1 leader_uuid: "655961c6e79944d29d4b4eab2d86aa5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655961c6e79944d29d4b4eab2d86aa5d" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37067 } } }
I20260806 06:39:50.303615 25702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:50.303534 25704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "655961c6e79944d29d4b4eab2d86aa5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655961c6e79944d29d4b4eab2d86aa5d" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37067 } } }
I20260806 06:39:50.303663 25704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:50.303988 25712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:50.304570 25712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:39:50.307272 25712 catalog_manager.cc:1347] Generated new cluster ID: b3047d4fdd394bb1983979ca4226a52a
I20260806 06:39:50.307339 25712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:39:50.308260 25701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.301652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56220 (local address 127.29.142.190:37067)
0806 06:39:50.301923 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:50.301931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:50.301956 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:39:50.302062 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:50.302068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:50.302130 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:50.302290 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:50.302532 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.304662 (+ 2130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.305839 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.306383 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.306530 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.307520 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:50.307525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:50.307525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:50.307713 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:50.307760 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:50.307773 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:50.307857 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:50.307939 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 06:39:50.308022 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":107,"threads_started":1}
I20260806 06:39:50.320190 25712 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:39:50.320938 25712 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:50.324640 25712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d: Generated new TSK 0
I20260806 06:39:50.324846 25712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:39:50.330426 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.129:0
--local_ip_for_outbound_sockets=127.29.142.129
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37067
--builtin_ntp_servers=127.29.142.148:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:39:50.420909 25728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:50.421190 25728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:50.421229 25728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:50.422420 25728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:50.422492 25728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:50.422533 25728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.129
I20260806 06:39:50.423985 25728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.142.129
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.129
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:50.424280 25728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:50.424611 25728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:50.430073 25735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:50.430073 25734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:50.430305 25728 server_base.cc:1034] running on GCE node
W20260806 06:39:50.430438 25738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:50.430722 25728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:50.431000 25728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:50.432171 25728 hybrid_clock.cc:648] HybridClock initialized: now 1785998390432144 us; error 44 us; skew 500 ppm
I20260806 06:39:50.433512 25728 webserver.cc:460] Webserver started at http://127.29.142.129:43923/ using document root <none> and password file <none>
I20260806 06:39:50.433796 25728 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:50.433872 25728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:50.434023 25728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:50.435245 25728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/data/instance:
uuid: "57b37d78bb484340890cb71e9db1f5a0"
format_stamp: "Formatted at 2026-08-06 06:39:50 on dist-test-slave-3kk6"
server_key: "589999ef93c52f99e38530645f265f48"
server_key_iv: "8bb594e43a96282534764e0b973e9ef7"
server_key_version: "encryptionkey@0"
I20260806 06:39:50.435631 25728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance:
uuid: "57b37d78bb484340890cb71e9db1f5a0"
format_stamp: "Formatted at 2026-08-06 06:39:50 on dist-test-slave-3kk6"
server_key: "589999ef93c52f99e38530645f265f48"
server_key_iv: "8bb594e43a96282534764e0b973e9ef7"
server_key_version: "encryptionkey@0"
I20260806 06:39:50.437044 25728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:39:50.437860 25750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:50.438093 25728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:50.438199 25728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/wal
uuid: "57b37d78bb484340890cb71e9db1f5a0"
format_stamp: "Formatted at 2026-08-06 06:39:50 on dist-test-slave-3kk6"
server_key: "589999ef93c52f99e38530645f265f48"
server_key_iv: "8bb594e43a96282534764e0b973e9ef7"
server_key_version: "encryptionkey@0"
I20260806 06:39:50.438275 25728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:50.449971 25728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:50.450346 25728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:50.450518 25728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:50.450870 25728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:50.451314 25728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:50.451375 25728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:50.451411 25728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:50.451433 25728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:50.457728 25728 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.129:35809
I20260806 06:39:50.457809 25910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.129:35809 every 8 connection(s)
I20260806 06:39:50.458083 25728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/data/info.pb
I20260806 06:39:50.463925 25701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.458955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.129:37205 (local address 127.29.142.190:37067)
0806 06:39:50.459011 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:50.459013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:50.459269 (+ 256us) server_negotiation.cc:407] Connection header received
0806 06:39:50.459829 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:50.459830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:50.459847 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:50.459907 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:50.460638 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.461160 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.462177 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.462391 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.462472 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.463558 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:50.463560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:50.463560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:50.463566 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:50.463588 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:50.463598 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:50.463673 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:50.463793 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 06:39:50.463833 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:39:50.464193 25916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.458935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37067 (local address 127.29.142.129:37205)
0806 06:39:50.459140 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:50.459175 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:50.459715 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:50.459980 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:50.459985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:50.459995 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:50.460533 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.460538 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.461269 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.461270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.462074 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.462076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.462484 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.462485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.462622 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.463365 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:39:50.463366 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:50.463398 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:39:50.463424 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:50.463694 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:50.463701 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:50.463706 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:39:50.463795 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 06:39:50.463914 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260806 06:39:50.464829 25911 heartbeater.cc:347] Connected to a master server at 127.29.142.190:37067
I20260806 06:39:50.464955 25911 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:50.465152 25911 heartbeater.cc:511] Master 127.29.142.190:37067 requested a full tablet report, sending...
I20260806 06:39:50.465710 25647 ts_manager.cc:194] Registered new tserver with Master: 57b37d78bb484340890cb71e9db1f5a0 (127.29.142.129:35809)
I20260806 06:39:50.466430 25647 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.142.129:37205
I20260806 06:39:50.467145 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 25728
I20260806 06:39:50.467242 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-0/wal/instance
I20260806 06:39:50.467453 30266 external_mini_cluster.cc:1428] Setting key 72b3b3c5b9ef05b3c9af1a4e750c7562
I20260806 06:39:50.468508 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.130:0
--local_ip_for_outbound_sockets=127.29.142.130
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37067
--builtin_ntp_servers=127.29.142.148:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:39:50.556720 25917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:50.556954 25917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:50.556979 25917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:50.558247 25917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:50.558305 25917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:50.558332 25917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.130
I20260806 06:39:50.559801 25917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.142.130
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.130
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:50.560084 25917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:50.560438 25917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:50.565838 25931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:50.565821 25927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:50.565956 25917 server_base.cc:1034] running on GCE node
W20260806 06:39:50.566146 25928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:50.566443 25917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:50.566807 25917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:50.567979 25917 hybrid_clock.cc:648] HybridClock initialized: now 1785998390567968 us; error 38 us; skew 500 ppm
I20260806 06:39:50.569550 25917 webserver.cc:460] Webserver started at http://127.29.142.130:38765/ using document root <none> and password file <none>
I20260806 06:39:50.569842 25917 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:50.569896 25917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:50.570070 25917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:50.571326 25917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/data/instance:
uuid: "41796b8560334463960ed78958345307"
format_stamp: "Formatted at 2026-08-06 06:39:50 on dist-test-slave-3kk6"
server_key: "2e5bfa926a6f0afc7b3b566fcd9b633c"
server_key_iv: "57f48aed3324fc406a7afdde1632fd51"
server_key_version: "encryptionkey@0"
I20260806 06:39:50.571741 25917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance:
uuid: "41796b8560334463960ed78958345307"
format_stamp: "Formatted at 2026-08-06 06:39:50 on dist-test-slave-3kk6"
server_key: "2e5bfa926a6f0afc7b3b566fcd9b633c"
server_key_iv: "57f48aed3324fc406a7afdde1632fd51"
server_key_version: "encryptionkey@0"
I20260806 06:39:50.573263 25917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:39:50.574213 25938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:50.574393 25917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:50.574496 25917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/wal
uuid: "41796b8560334463960ed78958345307"
format_stamp: "Formatted at 2026-08-06 06:39:50 on dist-test-slave-3kk6"
server_key: "2e5bfa926a6f0afc7b3b566fcd9b633c"
server_key_iv: "57f48aed3324fc406a7afdde1632fd51"
server_key_version: "encryptionkey@0"
I20260806 06:39:50.574560 25917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:50.586344 25917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:50.586759 25917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:50.586894 25917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:50.587110 25917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:50.587499 25917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:50.587550 25917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:50.587606 25917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:50.587634 25917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:50.593698 25917 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.130:42649
I20260806 06:39:50.593734 26089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.130:42649 every 8 connection(s)
I20260806 06:39:50.594151 25917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/data/info.pb
I20260806 06:39:50.594623 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 25917
I20260806 06:39:50.594794 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/wal/instance
I20260806 06:39:50.595067 30266 external_mini_cluster.cc:1428] Setting key 0471d0b8404520d651117c45e7b14916
I20260806 06:39:50.596153 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.142.131:0
--local_ip_for_outbound_sockets=127.29.142.131
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37067
--builtin_ntp_servers=127.29.142.148:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:39:50.601079 25701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.595108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.130:55875 (local address 127.29.142.190:37067)
0806 06:39:50.595184 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:50.595186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:50.595455 (+ 269us) server_negotiation.cc:407] Connection header received
0806 06:39:50.596092 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:50.596093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:50.596105 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:50.596190 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:50.596971 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.597759 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.599289 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.599624 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.599706 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.600728 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:50.600730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:50.600730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:50.600736 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:50.600756 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:50.600762 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:50.600829 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:50.600919 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:39:50.600953 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:39:50.601193 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.595079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37067 (local address 127.29.142.130:55875)
0806 06:39:50.595314 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:50.595368 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:50.595982 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:50.596294 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:50.596300 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:50.596314 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:50.596870 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.596874 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.597870 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.597871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.599201 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.599204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.599725 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.599726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.599865 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.600571 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:39:50.600572 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:50.600604 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:39:50.600630 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:50.600836 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:50.600841 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:50.600843 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:39:50.600900 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:39:50.600989 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260806 06:39:50.601787 26090 heartbeater.cc:347] Connected to a master server at 127.29.142.190:37067
I20260806 06:39:50.601910 26090 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:50.602105 26090 heartbeater.cc:511] Master 127.29.142.190:37067 requested a full tablet report, sending...
I20260806 06:39:50.602586 25647 ts_manager.cc:194] Registered new tserver with Master: 41796b8560334463960ed78958345307 (127.29.142.130:42649)
I20260806 06:39:50.602989 25647 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.142.130:55875
W20260806 06:39:50.683570 26093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:50.683818 26093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:50.683841 26093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:50.685072 26093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:50.685123 26093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:50.685149 26093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.142.131
I20260806 06:39:50.686551 26093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.142.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.142.131
--webserver_port=0
--tserver_master_addrs=127.29.142.190:37067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.142.131
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:50.686841 26093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:50.687182 26093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:50.692533 26102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:50.692685 26093 server_base.cc:1034] running on GCE node
W20260806 06:39:50.692543 26099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:50.692551 26107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:50.692994 26093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:50.693275 26093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:50.694430 26093 hybrid_clock.cc:648] HybridClock initialized: now 1785998390694407 us; error 37 us; skew 500 ppm
I20260806 06:39:50.695816 26093 webserver.cc:460] Webserver started at http://127.29.142.131:43653/ using document root <none> and password file <none>
I20260806 06:39:50.696081 26093 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:50.696131 26093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:50.696324 26093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:50.697471 26093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/data/instance:
uuid: "03d36bda0abe4f4a9695227caeddd15c"
format_stamp: "Formatted at 2026-08-06 06:39:50 on dist-test-slave-3kk6"
server_key: "beeddf9be15fd7a2d3d027dedfd2eeaf"
server_key_iv: "35b2023213146cd8ddd4fe1030402554"
server_key_version: "encryptionkey@0"
I20260806 06:39:50.697860 26093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance:
uuid: "03d36bda0abe4f4a9695227caeddd15c"
format_stamp: "Formatted at 2026-08-06 06:39:50 on dist-test-slave-3kk6"
server_key: "beeddf9be15fd7a2d3d027dedfd2eeaf"
server_key_iv: "35b2023213146cd8ddd4fe1030402554"
server_key_version: "encryptionkey@0"
I20260806 06:39:50.699342 26093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260806 06:39:50.700145 26116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:50.700331 26093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:50.700440 26093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/wal
uuid: "03d36bda0abe4f4a9695227caeddd15c"
format_stamp: "Formatted at 2026-08-06 06:39:50 on dist-test-slave-3kk6"
server_key: "beeddf9be15fd7a2d3d027dedfd2eeaf"
server_key_iv: "35b2023213146cd8ddd4fe1030402554"
server_key_version: "encryptionkey@0"
I20260806 06:39:50.700529 26093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:50.712474 26093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:50.712875 26093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:50.713060 26093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:50.713346 26093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:39:50.713714 26093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:39:50.713774 26093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:50.713822 26093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:39:50.713846 26093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:50.719986 26093 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.131:40533
I20260806 06:39:50.720064 26268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.131:40533 every 8 connection(s)
I20260806 06:39:50.720432 26093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/data/info.pb
I20260806 06:39:50.722922 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 26093
I20260806 06:39:50.723093 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/wal/instance
I20260806 06:39:50.723317 30266 external_mini_cluster.cc:1428] Setting key 94c7f5b1cb75fd88f9fa0df4f5f8c485
I20260806 06:39:50.727051 25701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.721371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:50973 (local address 127.29.142.190:37067)
0806 06:39:50.721450 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:50.721452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:50.721745 (+ 293us) server_negotiation.cc:407] Connection header received
0806 06:39:50.722336 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:50.722338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:50.722354 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:50.722417 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:50.723524 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.724276 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.725401 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.725596 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.725658 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.726773 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:50.726774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:50.726774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:50.726780 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:50.726796 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:50.726800 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:50.726853 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:50.726945 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:39:50.726978 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:39:50.727257 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.721335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37067 (local address 127.29.142.131:50973)
0806 06:39:50.721575 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:50.721614 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:50.722177 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:50.722499 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:50.722502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:50.722513 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:39:50.723331 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.723340 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.724384 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.724385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.725311 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.725313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.725723 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.725724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.725853 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.726556 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:39:50.726557 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:39:50.726593 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:39:50.726622 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:39:50.726868 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:39:50.726873 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:39:50.726875 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:39:50.726933 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:39:50.727030 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":109,"threads_started":1}
I20260806 06:39:50.727995 26270 heartbeater.cc:347] Connected to a master server at 127.29.142.190:37067
I20260806 06:39:50.728124 26270 heartbeater.cc:464] Registering TS with master...
I20260806 06:39:50.728338 26270 heartbeater.cc:511] Master 127.29.142.190:37067 requested a full tablet report, sending...
I20260806 06:39:50.728793 25647 ts_manager.cc:194] Registered new tserver with Master: 03d36bda0abe4f4a9695227caeddd15c (127.29.142.131:40533)
I20260806 06:39:50.729194 25647 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.142.131:50973
I20260806 06:39:50.735718 30266 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:39:50.740594 25701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.737289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56222 (local address 127.29.142.190:37067)
0806 06:39:50.737404 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:50.737406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:50.737488 (+ 82us) server_negotiation.cc:407] Connection header received
0806 06:39:50.737536 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:50.737537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:50.737551 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:50.737604 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:50.737767 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.738278 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.739130 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.739345 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.739427 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.740259 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:50.740260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:50.740261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:50.740266 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:50.740281 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:50.740285 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:50.740352 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:50.740409 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 06:39:50.740439 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 06:39:50.742936 25647 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:39:50.744724 25647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:39:50.751993 26283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.747369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:42649 (local address 127.0.0.1:53064)
0806 06:39:50.747515 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:50.747564 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:50.747645 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:50.748478 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:50.748479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:50.748480 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:50.748554 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.748556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.749471 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.749472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.750920 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.750922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.751670 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.751671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.751756 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.751769 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:39:50.751849 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 06:39:50.751908 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":34,"thread_start_us":75,"threads_started":1}
I20260806 06:39:50.752014 26288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.747593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53064 (local address 127.29.142.130:42649)
0806 06:39:50.748259 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:50.748264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:50.748274 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:39:50.748314 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:50.748316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:50.748332 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:50.748406 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:50.748654 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.749368 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.751037 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.751542 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.751663 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.751783 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:50.751864 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 06:39:50.751903 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":106,"threads_started":1}
I20260806 06:39:50.752185 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.747606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.131:40533 (local address 127.0.0.1:48634)
0806 06:39:50.748027 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:50.748036 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:50.748120 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:50.748473 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:50.748474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:50.748475 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:50.748555 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.748556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.749747 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.749748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.750919 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.750922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.751893 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.751894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.751965 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.751969 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:39:50.752023 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:39:50.752072 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":12,"thread_start_us":67,"threads_started":1}
I20260806 06:39:50.752411 26285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.747460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36610 (local address 127.29.142.129:35809)
0806 06:39:50.747735 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:50.747741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:50.747929 (+ 188us) server_negotiation.cc:407] Connection header received
0806 06:39:50.748016 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:50.748019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:50.748038 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:50.748127 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:50.748308 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.749501 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.750896 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.751667 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.751753 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.751847 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:50.752249 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 06:39:50.752311 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":106,"threads_started":1}
I20260806 06:39:50.752430 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.747843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48634 (local address 127.29.142.131:40533)
0806 06:39:50.748022 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:50.748026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:50.748329 (+ 303us) server_negotiation.cc:407] Connection header received
0806 06:39:50.748377 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:50.748379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:50.748395 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:50.748473 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:50.748618 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.749660 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.750999 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.751708 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.751773 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.751857 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:50.752300 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 06:39:50.752346 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
I20260806 06:39:50.752586 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:50.747370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:35809 (local address 127.0.0.1:36610)
0806 06:39:50.747839 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:50.747848 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:50.747926 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:50.748143 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:50.748147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:50.748149 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:50.748234 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.748237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.749605 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.749606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.750803 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:50.750806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:50.751891 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:50.751893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:50.751965 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:50.751970 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:39:50.752023 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:39:50.752512 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":476,"spinlock_wait_cycles":1792,"thread_start_us":142,"threads_started":1}
I20260806 06:39:50.753101 26182 tablet_service.cc:1467] Processing CreateTablet for tablet 07c8a977902648ba8278c9875544315c (DEFAULT_TABLE table=test-table [id=f9b491e672134bb8941f20afa018d0ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:50.753140 26001 tablet_service.cc:1467] Processing CreateTablet for tablet 07c8a977902648ba8278c9875544315c (DEFAULT_TABLE table=test-table [id=f9b491e672134bb8941f20afa018d0ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:50.753496 26001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07c8a977902648ba8278c9875544315c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:50.753504 26182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07c8a977902648ba8278c9875544315c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:50.756147 25812 tablet_service.cc:1467] Processing CreateTablet for tablet 07c8a977902648ba8278c9875544315c (DEFAULT_TABLE table=test-table [id=f9b491e672134bb8941f20afa018d0ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:39:50.756573 25812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07c8a977902648ba8278c9875544315c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:50.756711 26292 tablet_bootstrap.cc:492] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307: Bootstrap starting.
I20260806 06:39:50.757393 26292 tablet_bootstrap.cc:654] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:50.757711 26292 log.cc:824] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:50.758514 26292 tablet_bootstrap.cc:492] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307: No bootstrap required, opened a new log
I20260806 06:39:50.758606 26292 ts_tablet_manager.cc:1397] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:39:50.759768 26294 tablet_bootstrap.cc:492] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0: Bootstrap starting.
I20260806 06:39:50.759948 26291 tablet_bootstrap.cc:492] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c: Bootstrap starting.
I20260806 06:39:50.760499 26294 tablet_bootstrap.cc:654] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:50.760596 26291 tablet_bootstrap.cc:654] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:50.760881 26294 log.cc:824] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:50.760914 26291 log.cc:824] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:50.761287 26292 raft_consensus.cc:348] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41796b8560334463960ed78958345307" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42649 } } peers { permanent_uuid: "57b37d78bb484340890cb71e9db1f5a0" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35809 } } peers { permanent_uuid: "03d36bda0abe4f4a9695227caeddd15c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 40533 } }
I20260806 06:39:50.761512 26292 raft_consensus.cc:374] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:50.761569 26292 raft_consensus.cc:729] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41796b8560334463960ed78958345307, State: Initialized, Role: FOLLOWER
I20260806 06:39:50.761718 26291 tablet_bootstrap.cc:492] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c: No bootstrap required, opened a new log
I20260806 06:39:50.761696 26292 consensus_queue.cc:260] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41796b8560334463960ed78958345307" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42649 } } peers { permanent_uuid: "57b37d78bb484340890cb71e9db1f5a0" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35809 } } peers { permanent_uuid: "03d36bda0abe4f4a9695227caeddd15c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 40533 } }
I20260806 06:39:50.761780 26291 ts_tablet_manager.cc:1397] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:39:50.762036 26292 ts_tablet_manager.cc:1428] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:39:50.762095 26090 heartbeater.cc:503] Master 127.29.142.190:37067 was elected leader, sending a full tablet report...
I20260806 06:39:50.763110 26294 tablet_bootstrap.cc:492] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0: No bootstrap required, opened a new log
I20260806 06:39:50.763202 26294 ts_tablet_manager.cc:1397] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:39:50.764770 26291 raft_consensus.cc:348] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41796b8560334463960ed78958345307" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42649 } } peers { permanent_uuid: "57b37d78bb484340890cb71e9db1f5a0" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35809 } } peers { permanent_uuid: "03d36bda0abe4f4a9695227caeddd15c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 40533 } }
I20260806 06:39:50.764961 26291 raft_consensus.cc:374] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:50.765000 26291 raft_consensus.cc:729] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03d36bda0abe4f4a9695227caeddd15c, State: Initialized, Role: FOLLOWER
I20260806 06:39:50.765138 26291 consensus_queue.cc:260] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41796b8560334463960ed78958345307" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42649 } } peers { permanent_uuid: "57b37d78bb484340890cb71e9db1f5a0" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35809 } } peers { permanent_uuid: "03d36bda0abe4f4a9695227caeddd15c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 40533 } }
I20260806 06:39:50.765446 26291 ts_tablet_manager.cc:1428] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 06:39:50.765499 26270 heartbeater.cc:503] Master 127.29.142.190:37067 was elected leader, sending a full tablet report...
I20260806 06:39:50.765787 26294 raft_consensus.cc:348] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41796b8560334463960ed78958345307" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42649 } } peers { permanent_uuid: "57b37d78bb484340890cb71e9db1f5a0" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35809 } } peers { permanent_uuid: "03d36bda0abe4f4a9695227caeddd15c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 40533 } }
I20260806 06:39:50.765915 26294 raft_consensus.cc:374] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:50.765975 26294 raft_consensus.cc:729] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b37d78bb484340890cb71e9db1f5a0, State: Initialized, Role: FOLLOWER
I20260806 06:39:50.766129 26294 consensus_queue.cc:260] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41796b8560334463960ed78958345307" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42649 } } peers { permanent_uuid: "57b37d78bb484340890cb71e9db1f5a0" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35809 } } peers { permanent_uuid: "03d36bda0abe4f4a9695227caeddd15c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 40533 } }
I20260806 06:39:50.766480 26294 ts_tablet_manager.cc:1428] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:39:50.766538 25911 heartbeater.cc:503] Master 127.29.142.190:37067 was elected leader, sending a full tablet report...
W20260806 06:39:50.844800 26091 tablet.cc:2369] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:39:50.958873 25913 tablet.cc:2369] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:39:50.971087 26272 tablet.cc:2369] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:39:51.019205 26301 raft_consensus.cc:482] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:39:51.019366 26301 raft_consensus.cc:504] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41796b8560334463960ed78958345307" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42649 } } peers { permanent_uuid: "57b37d78bb484340890cb71e9db1f5a0" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35809 } } peers { permanent_uuid: "03d36bda0abe4f4a9695227caeddd15c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 40533 } }
I20260806 06:39:51.019851 26301 leader_election.cc:288] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41796b8560334463960ed78958345307 (127.29.142.130:42649), 57b37d78bb484340890cb71e9db1f5a0 (127.29.142.129:35809)
I20260806 06:39:51.022934 26285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:51.020278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:36831 (local address 127.29.142.129:35809)
0806 06:39:51.020328 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:51.020330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:51.020339 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:39:51.020376 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:51.020377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:51.020386 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:51.020458 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:51.020637 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:51.021127 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:51.022146 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:51.022510 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:51.022600 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:51.022610 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:51.022819 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 06:39:51.022856 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:39:51.022933 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:51.020092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:35809 (local address 127.29.142.131:36831)
0806 06:39:51.020186 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:51.020201 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:51.020264 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:51.020465 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:51.020466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:51.020467 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:51.020551 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:51.020554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:51.021276 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:51.021277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:51.022043 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:51.022045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:51.022600 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:51.022601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:51.022726 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:51.022730 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:39:51.022799 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:39:51.022856 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 06:39:51.023038 26288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:51.020179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.142.131:33335 (local address 127.29.142.130:42649)
0806 06:39:51.020282 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:51.020284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:51.020466 (+ 182us) server_negotiation.cc:407] Connection header received
0806 06:39:51.020546 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:51.020547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:51.020557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:51.020634 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:39:51.020812 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:51.021279 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:51.022222 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:51.022593 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:51.022746 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:51.022755 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:51.022923 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 06:39:51.022965 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 06:39:51.023058 26303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:51.020102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.130:42649 (local address 127.29.142.131:33335)
0806 06:39:51.020379 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:51.020389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:39:51.020477 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:39:51.020637 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:39:51.020638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:39:51.020640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:39:51.020723 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:51.020725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:51.021371 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:51.021372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:51.022112 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:39:51.022114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:39:51.022751 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:39:51.022752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:39:51.022834 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:51.022838 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:39:51.022903 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:39:51.022944 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":73,"thread_start_us":139,"threads_started":1}
I20260806 06:39:51.023360 25844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07c8a977902648ba8278c9875544315c" candidate_uuid: "03d36bda0abe4f4a9695227caeddd15c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b37d78bb484340890cb71e9db1f5a0" is_pre_election: true
I20260806 06:39:51.023479 26027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07c8a977902648ba8278c9875544315c" candidate_uuid: "03d36bda0abe4f4a9695227caeddd15c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41796b8560334463960ed78958345307" is_pre_election: true
I20260806 06:39:51.023569 25844 raft_consensus.cc:2454] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03d36bda0abe4f4a9695227caeddd15c in term 0.
I20260806 06:39:51.023674 26027 raft_consensus.cc:2454] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03d36bda0abe4f4a9695227caeddd15c in term 0.
I20260806 06:39:51.023851 26119 leader_election.cc:302] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03d36bda0abe4f4a9695227caeddd15c, 57b37d78bb484340890cb71e9db1f5a0; no voters:
I20260806 06:39:51.024013 26301 raft_consensus.cc:2781] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:39:51.024075 26301 raft_consensus.cc:482] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:39:51.024115 26301 raft_consensus.cc:3037] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:51.024881 26301 raft_consensus.cc:504] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41796b8560334463960ed78958345307" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42649 } } peers { permanent_uuid: "57b37d78bb484340890cb71e9db1f5a0" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35809 } } peers { permanent_uuid: "03d36bda0abe4f4a9695227caeddd15c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 40533 } }
I20260806 06:39:51.025041 26301 leader_election.cc:288] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [CANDIDATE]: Term 1 election: Requested vote from peers 41796b8560334463960ed78958345307 (127.29.142.130:42649), 57b37d78bb484340890cb71e9db1f5a0 (127.29.142.129:35809)
I20260806 06:39:51.025209 26027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07c8a977902648ba8278c9875544315c" candidate_uuid: "03d36bda0abe4f4a9695227caeddd15c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41796b8560334463960ed78958345307"
I20260806 06:39:51.025235 25844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07c8a977902648ba8278c9875544315c" candidate_uuid: "03d36bda0abe4f4a9695227caeddd15c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57b37d78bb484340890cb71e9db1f5a0"
I20260806 06:39:51.025314 25844 raft_consensus.cc:3037] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:51.025314 26027 raft_consensus.cc:3037] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:51.026304 25844 raft_consensus.cc:2454] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03d36bda0abe4f4a9695227caeddd15c in term 1.
I20260806 06:39:51.026304 26027 raft_consensus.cc:2454] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03d36bda0abe4f4a9695227caeddd15c in term 1.
I20260806 06:39:51.026546 26122 leader_election.cc:302] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03d36bda0abe4f4a9695227caeddd15c, 41796b8560334463960ed78958345307; no voters:
I20260806 06:39:51.026703 26301 raft_consensus.cc:2781] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:51.026929 26301 raft_consensus.cc:686] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 1 LEADER]: Becoming Leader. State: Replica: 03d36bda0abe4f4a9695227caeddd15c, State: Running, Role: LEADER
I20260806 06:39:51.027050 26301 consensus_queue.cc:237] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41796b8560334463960ed78958345307" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42649 } } peers { permanent_uuid: "57b37d78bb484340890cb71e9db1f5a0" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35809 } } peers { permanent_uuid: "03d36bda0abe4f4a9695227caeddd15c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 40533 } }
I20260806 06:39:51.028019 25647 catalog_manager.cc:5495] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c reported cstate change: term changed from 0 to 1, leader changed from <none> to 03d36bda0abe4f4a9695227caeddd15c (127.29.142.131). New cstate: current_term: 1 leader_uuid: "03d36bda0abe4f4a9695227caeddd15c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41796b8560334463960ed78958345307" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57b37d78bb484340890cb71e9db1f5a0" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03d36bda0abe4f4a9695227caeddd15c" member_type: VOTER last_known_addr { host: "127.29.142.131" port: 40533 } health_report { overall_health: HEALTHY } } }
I20260806 06:39:51.035590 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:51.033178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48648 (local address 127.29.142.131:40533)
0806 06:39:51.033253 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:51.033255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:51.033264 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:39:51.033309 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:51.033310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:51.033328 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:51.033389 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:39:51.033526 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:51.034030 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:51.034850 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:51.035037 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:51.035095 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:51.035228 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:39:51.035334 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:39:51.035397 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:51.035418 (+ 21us) server_negotiation.cc:299] Negotiation successful
0806 06:39:51.035434 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:39:51.037699 25844 raft_consensus.cc:1261] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0 [term 1 FOLLOWER]: Refusing update from remote peer 03d36bda0abe4f4a9695227caeddd15c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:51.037699 26027 raft_consensus.cc:1261] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307 [term 1 FOLLOWER]: Refusing update from remote peer 03d36bda0abe4f4a9695227caeddd15c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:39:51.038183 26304 consensus_queue.cc:1035] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [LEADER]: Connected to new peer: Peer: permanent_uuid: "41796b8560334463960ed78958345307" member_type: VOTER last_known_addr { host: "127.29.142.130" port: 42649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:51.038364 26301 consensus_queue.cc:1035] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [LEADER]: Connected to new peer: Peer: permanent_uuid: "57b37d78bb484340890cb71e9db1f5a0" member_type: VOTER last_known_addr { host: "127.29.142.129" port: 35809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:39:51.041430 26311 mvcc.cc:205] Tried to move back new op lower bound from 7315449409687019520 to 7315449409647472640. Current Snapshot: MvccSnapshot[applied={T|T < 7315449409687019520 or (T in {7315449409687019520})}]
I20260806 06:39:51.043546 25646 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:51.043700 25646 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:39:51.044570 25646 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 655961c6e79944d29d4b4eab2d86aa5d: Sending DeleteTablet for 3 replicas of tablet 07c8a977902648ba8278c9875544315c
I20260806 06:39:51.044960 25812 tablet_service.cc:1514] Processing DeleteTablet for tablet 07c8a977902648ba8278c9875544315c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:51 UTC) from {username='slave'} at 127.0.0.1:36610
I20260806 06:39:51.044938 26001 tablet_service.cc:1514] Processing DeleteTablet for tablet 07c8a977902648ba8278c9875544315c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:51 UTC) from {username='slave'} at 127.0.0.1:53064
I20260806 06:39:51.044987 25811 tablet_service.cc:1514] Processing DeleteTablet for tablet 07c8a977902648ba8278c9875544315c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:51 UTC) from {username='slave'} at 127.0.0.1:36610
I20260806 06:39:51.045198 26182 tablet_service.cc:1514] Processing DeleteTablet for tablet 07c8a977902648ba8278c9875544315c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:39:51 UTC) from {username='slave'} at 127.0.0.1:48634
I20260806 06:39:51.045244 26321 tablet_replica.cc:330] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307: stopping tablet replica
I20260806 06:39:51.045356 26321 raft_consensus.cc:2229] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:39:51.045475 26322 tablet_replica.cc:330] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c: stopping tablet replica
I20260806 06:39:51.045486 26321 raft_consensus.cc:2258] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:51.045591 26322 raft_consensus.cc:2229] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:39:51.045660 26319 tablet_replica.cc:330] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0: stopping tablet replica
I20260806 06:39:51.045751 26322 raft_consensus.cc:2258] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:39:51.045751 26319 raft_consensus.cc:2229] T 07c8a977902648ba8278c9875544315c P 57b37d78bb484340890cb71e9db1f5a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:39:51.045776 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 25728
I20260806 06:39:51.046815 26322 ts_tablet_manager.cc:1905] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:51.049181 26322 ts_tablet_manager.cc:1918] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:51.049188 26321 ts_tablet_manager.cc:1905] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:39:51.049271 26322 log.cc:1196] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-2/wal/wals/07c8a977902648ba8278c9875544315c
I20260806 06:39:51.049561 26322 ts_tablet_manager.cc:1939] T 07c8a977902648ba8278c9875544315c P 03d36bda0abe4f4a9695227caeddd15c: Deleting consensus metadata
I20260806 06:39:51.050552 26321 ts_tablet_manager.cc:1918] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:39:51.050661 26321 log.cc:1196] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307: Deleting WAL directory at /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998137760399-30266-0/minicluster-data/ts-1/wal/wals/07c8a977902648ba8278c9875544315c
I20260806 06:39:51.051040 26321 ts_tablet_manager.cc:1939] T 07c8a977902648ba8278c9875544315c P 41796b8560334463960ed78958345307: Deleting consensus metadata
I20260806 06:39:51.051587 25627 catalog_manager.cc:4841] TS 41796b8560334463960ed78958345307 (127.29.142.130:42649): tablet 07c8a977902648ba8278c9875544315c (table test-table [id=f9b491e672134bb8941f20afa018d0ab]) successfully deleted
I20260806 06:39:51.052659 25627 catalog_manager.cc:4841] TS 03d36bda0abe4f4a9695227caeddd15c (127.29.142.131:40533): tablet 07c8a977902648ba8278c9875544315c (table test-table [id=f9b491e672134bb8941f20afa018d0ab]) successfully deleted
W20260806 06:39:51.058358 25624 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.142.129:35809 (error 108)
I20260806 06:39:51.058665 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 25917
I20260806 06:39:51.059058 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:51.058860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.129:35809 (local address 127.0.0.1:36618)
0806 06:39:51.058973 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 06:39:51.059017 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.129:35809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260806 06:39:51.060994 25624 catalog_manager.cc:4584] TS 57b37d78bb484340890cb71e9db1f5a0 (127.29.142.129:35809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 07c8a977902648ba8278c9875544315c: Network error: Client connection negotiation failed: client connection to 127.29.142.129:35809: connect: Connection refused (error 111)
I20260806 06:39:51.067912 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 26093
I20260806 06:39:51.080895 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 25601
2026-08-06T06:39:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (953 ms)
[----------] 6 tests from EncryptionPolicyTest (9552 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:39:51 dist-test-slave-3kk6 krb5kdc[26328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:39:51 dist-test-slave-3kk6 krb5kdc[26328](info): set up 2 sockets
Aug 06 06:39:51 dist-test-slave-3kk6 krb5kdc[26328](info): commencing operation
krb5kdc: starting...
W20260806 06:39:52.308605 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:39:52 dist-test-slave-3kk6 krb5kdc[26328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:39:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:39:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:39:52.442458 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33781
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:52.529819 26348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:52.530050 26348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:52.530074 26348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:52.531355 26348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:52.531407 26348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:52.531422 26348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:52.531435 26348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:52.531447 26348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:52.533047 26348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:52.533329 26348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:52.533608 26348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:52.538985 26355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:52.538991 26354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:52.539122 26348 server_base.cc:1034] running on GCE node
W20260806 06:39:52.538991 26358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:52.539559 26348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:52.539958 26348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:52.541169 26348 hybrid_clock.cc:648] HybridClock initialized: now 1785998392541152 us; error 43 us; skew 500 ppm
Aug 06 06:39:52 dist-test-slave-3kk6 krb5kdc[26328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998392, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:52.544255 26348 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:39:52.544763 26348 webserver.cc:460] Webserver started at http://127.0.0.1:34093/ using document root <none> and password file <none>
I20260806 06:39:52.545058 26348 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:52.545140 26348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:52.545305 26348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:52.546514 26348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "dfaf7a38c3a64131ab5869c8f4131c38"
format_stamp: "Formatted at 2026-08-06 06:39:52 on dist-test-slave-3kk6"
server_key: "d2208052e0052ab9a08abeef7b628657"
server_key_iv: "f586731f7a8c5cf34bc44f8ff2fa8bc4"
server_key_version: "encryptionkey@0"
I20260806 06:39:52.546989 26348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "dfaf7a38c3a64131ab5869c8f4131c38"
format_stamp: "Formatted at 2026-08-06 06:39:52 on dist-test-slave-3kk6"
server_key: "d2208052e0052ab9a08abeef7b628657"
server_key_iv: "f586731f7a8c5cf34bc44f8ff2fa8bc4"
server_key_version: "encryptionkey@0"
I20260806 06:39:52.548513 26348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:52.549463 26373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:52.549686 26348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:52.549844 26348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "dfaf7a38c3a64131ab5869c8f4131c38"
format_stamp: "Formatted at 2026-08-06 06:39:52 on dist-test-slave-3kk6"
server_key: "d2208052e0052ab9a08abeef7b628657"
server_key_iv: "f586731f7a8c5cf34bc44f8ff2fa8bc4"
server_key_version: "encryptionkey@0"
I20260806 06:39:52.549938 26348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:52.564723 26348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:52.567571 26348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:52.567781 26348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:52.572372 26348 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33781
I20260806 06:39:52.572408 26443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33781 every 8 connection(s)
I20260806 06:39:52.572835 26348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:52.573789 26444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:52.576402 26444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38: Bootstrap starting.
I20260806 06:39:52.577049 26444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:52.577329 26444 log.cc:824] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:52.578120 26444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38: No bootstrap required, opened a new log
I20260806 06:39:52.578919 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 26348
I20260806 06:39:52.579063 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:39:52.579262 30266 external_mini_cluster.cc:1428] Setting key f80aaa78ca2f00938aa094c55148ac7d
I20260806 06:39:52.580410 26444 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaf7a38c3a64131ab5869c8f4131c38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33781 } }
I20260806 06:39:52.580588 26444 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:52.580658 26444 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfaf7a38c3a64131ab5869c8f4131c38, State: Initialized, Role: FOLLOWER
I20260806 06:39:52.580802 26444 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaf7a38c3a64131ab5869c8f4131c38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33781 } }
I20260806 06:39:52.580888 26444 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:52.580914 26444 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:52.580986 26444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:52.582104 26444 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaf7a38c3a64131ab5869c8f4131c38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33781 } }
I20260806 06:39:52.582249 26444 leader_election.cc:302] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfaf7a38c3a64131ab5869c8f4131c38; no voters:
I20260806 06:39:52.582485 26444 leader_election.cc:288] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:39:52.582520 26451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:52.582671 26451 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [term 1 LEADER]: Becoming Leader. State: Replica: dfaf7a38c3a64131ab5869c8f4131c38, State: Running, Role: LEADER
I20260806 06:39:52.582821 26444 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:52.582839 26451 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaf7a38c3a64131ab5869c8f4131c38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33781 } }
I20260806 06:39:52.583187 26451 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfaf7a38c3a64131ab5869c8f4131c38. Latest consensus state: current_term: 1 leader_uuid: "dfaf7a38c3a64131ab5869c8f4131c38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaf7a38c3a64131ab5869c8f4131c38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33781 } } }
I20260806 06:39:52.583295 26451 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:52.583204 26452 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfaf7a38c3a64131ab5869c8f4131c38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfaf7a38c3a64131ab5869c8f4131c38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33781 } } }
I20260806 06:39:52.583475 26452 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38 [sys.catalog]: This master's current role is: LEADER
W20260806 06:39:52.584863 26472 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:39:52.584924 26472 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:39:52.584985 26458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:52.585155 26458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:39:52 dist-test-slave-3kk6 krb5kdc[26328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:39:52.587579 26458 catalog_manager.cc:1347] Generated new cluster ID: bde91cbffa734cdfb62b6d528a5182b0
I20260806 06:39:52.587644 26458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:39:52.591778 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:52.579938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52451 (local address 127.0.0.1:33781)
0806 06:39:52.580163 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:52.580172 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:52.580201 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:39:52.580361 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:52.580367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:52.580413 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:52.580559 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:52.581385 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:52.582390 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:52.583518 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:52.584066 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:52.584133 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:52.587381 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:52.587385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:52.587386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:52.587400 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:52.589716 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:52.590277 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:52.590281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:52.590281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:52.590427 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:52.590756 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:52.590757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:52.590757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:52.591094 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:52.591275 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:52.591426 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 06:39:52.591545 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":109,"threads_started":1}
I20260806 06:39:52.593075 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:39:52.602635 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:52.594614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45109 (local address 127.0.0.1:33781)
0806 06:39:52.594678 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:52.594680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:52.594690 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:39:52.594752 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:52.594753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:52.594764 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:52.594828 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:52.595475 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:52.596284 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:52.597291 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:52.597636 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:52.597713 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:52.599321 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:52.599323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:52.599324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:52.599332 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:52.601156 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:52.601618 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:52.601620 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:52.601620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:52.601689 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:52.601928 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:52.601928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:52.601929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:52.602075 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:52.602186 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:52.602332 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 06:39:52.602409 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:39:52.606900 26458 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:39:52.607477 26458 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:52.609616 26458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfaf7a38c3a64131ab5869c8f4131c38: Generated new TSK 0
I20260806 06:39:52.609827 26458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:39:52.615322 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 26348
W20260806 06:39:52.625634 26325 connection.cc:535] client connection to 127.0.0.1:33781 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33781): Connection reset by peer (error 104)
2026-08-06T06:39:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1530 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:39:52 dist-test-slave-3kk6 krb5kdc[26491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:39:52 dist-test-slave-3kk6 krb5kdc[26491](info): set up 2 sockets
Aug 06 06:39:52 dist-test-slave-3kk6 krb5kdc[26491](info): commencing operation
krb5kdc: starting...
W20260806 06:39:53.844205 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:39:53 dist-test-slave-3kk6 krb5kdc[26491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:39:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:39:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:39:53.971575 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33577
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:54.058142 26511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:54.058405 26511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:54.058434 26511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:54.059849 26511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:54.059912 26511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:54.059927 26511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:54.059940 26511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:54.059954 26511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:54.061579 26511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33577
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:54.061877 26511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:54.062130 26511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:54.067374 26523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:54.067368 26517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:54.067371 26520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:54.067718 26511 server_base.cc:1034] running on GCE node
I20260806 06:39:54.067982 26511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:54.068291 26511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:54.069464 26511 hybrid_clock.cc:648] HybridClock initialized: now 1785998394069454 us; error 30 us; skew 500 ppm
Aug 06 06:39:54 dist-test-slave-3kk6 krb5kdc[26491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998394, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:54.072162 26511 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:39:54.072629 26511 webserver.cc:460] Webserver started at http://127.0.0.1:37355/ using document root <none> and password file <none>
I20260806 06:39:54.072885 26511 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:54.072958 26511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:54.073113 26511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:54.074244 26511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "7ede8d378c2a40e3aa662fe34e2aa6f3"
format_stamp: "Formatted at 2026-08-06 06:39:54 on dist-test-slave-3kk6"
server_key: "b10174a6ad07b0042616d7f6c81f16af"
server_key_iv: "f7faef58bfc4626397a33fbf99c3419d"
server_key_version: "encryptionkey@0"
I20260806 06:39:54.074695 26511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "7ede8d378c2a40e3aa662fe34e2aa6f3"
format_stamp: "Formatted at 2026-08-06 06:39:54 on dist-test-slave-3kk6"
server_key: "b10174a6ad07b0042616d7f6c81f16af"
server_key_iv: "f7faef58bfc4626397a33fbf99c3419d"
server_key_version: "encryptionkey@0"
I20260806 06:39:54.076128 26511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:54.076930 26530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:54.077118 26511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:54.077229 26511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "7ede8d378c2a40e3aa662fe34e2aa6f3"
format_stamp: "Formatted at 2026-08-06 06:39:54 on dist-test-slave-3kk6"
server_key: "b10174a6ad07b0042616d7f6c81f16af"
server_key_iv: "f7faef58bfc4626397a33fbf99c3419d"
server_key_version: "encryptionkey@0"
I20260806 06:39:54.077318 26511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:54.089149 26511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:54.091179 26511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:54.091361 26511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:54.095899 26511 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33577
I20260806 06:39:54.096148 26598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33577 every 8 connection(s)
I20260806 06:39:54.096480 26511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:54.097316 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 26511
I20260806 06:39:54.097448 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:39:54.097657 30266 external_mini_cluster.cc:1428] Setting key 9b2b5e8c872d9a2e0c3cfddce2353c85
I20260806 06:39:54.100198 26599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:54.103744 26599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3: Bootstrap starting.
I20260806 06:39:54.104568 26599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:54.104962 26599 log.cc:824] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:54.105577 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:54.098344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49629 (local address 127.0.0.1:33577)
0806 06:39:54.099050 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:54.099060 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:54.099095 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:39:54.099547 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:54.099554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:54.099628 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:54.099791 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:54.100193 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:54.101498 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:54.102716 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:54.103387 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:54.103510 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:54.104636 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:54.104641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:54.104641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:54.104898 (+ 257us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:54.104941 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:54.104956 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:54.105067 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:54.105251 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 06:39:54.105315 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":600,"threads_started":1}
I20260806 06:39:54.105903 26599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3: No bootstrap required, opened a new log
I20260806 06:39:54.106705 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:39:54.108835 26599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ede8d378c2a40e3aa662fe34e2aa6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33577 } }
I20260806 06:39:54.109025 26599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:54.109053 26599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ede8d378c2a40e3aa662fe34e2aa6f3, State: Initialized, Role: FOLLOWER
I20260806 06:39:54.109198 26599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ede8d378c2a40e3aa662fe34e2aa6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33577 } }
I20260806 06:39:54.109272 26599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:54.109328 26599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:54.109383 26599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:54.110216 26599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ede8d378c2a40e3aa662fe34e2aa6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33577 } }
I20260806 06:39:54.110332 26599 leader_election.cc:302] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ede8d378c2a40e3aa662fe34e2aa6f3; no voters:
I20260806 06:39:54.110494 26599 leader_election.cc:288] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:39:54.110626 26613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:54.110782 26599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:54.110868 26613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [term 1 LEADER]: Becoming Leader. State: Replica: 7ede8d378c2a40e3aa662fe34e2aa6f3, State: Running, Role: LEADER
I20260806 06:39:54.111008 26613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ede8d378c2a40e3aa662fe34e2aa6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33577 } }
I20260806 06:39:54.111410 26617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ede8d378c2a40e3aa662fe34e2aa6f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ede8d378c2a40e3aa662fe34e2aa6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33577 } } }
I20260806 06:39:54.111508 26617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:54.111743 26626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:54.111683 26617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ede8d378c2a40e3aa662fe34e2aa6f3. Latest consensus state: current_term: 1 leader_uuid: "7ede8d378c2a40e3aa662fe34e2aa6f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ede8d378c2a40e3aa662fe34e2aa6f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33577 } } }
I20260806 06:39:54.111862 26617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:54.112422 26626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:39:54.113132 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:54.108469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45037 (local address 127.0.0.1:33577)
0806 06:39:54.108515 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:54.108517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:54.108610 (+ 93us) server_negotiation.cc:407] Connection header received
0806 06:39:54.108673 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:54.108674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:54.108682 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:54.108729 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:54.108870 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:54.109554 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:54.110580 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:54.110936 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:54.111016 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:54.112686 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:54.112687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:54.112688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:54.112693 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:54.112709 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:54.112715 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:54.112774 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:54.113024 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 06:39:54.113034 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:39:54.114079 26626 catalog_manager.cc:1347] Generated new cluster ID: 516d4436f0cd4508bb6870beb94003fa
I20260806 06:39:54.114131 26626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:39:54.131636 26626 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:39:54.132231 26626 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:54.136093 26626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ede8d378c2a40e3aa662fe34e2aa6f3: Generated new TSK 0
I20260806 06:39:54.136292 26626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:39:54.141373 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 26511
W20260806 06:39:54.153825 26489 connection.cc:535] client connection to 127.0.0.1:33577 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33577): Connection reset by peer (error 104)
2026-08-06T06:39:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1527 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:39:54 dist-test-slave-3kk6 krb5kdc[26641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:39:54 dist-test-slave-3kk6 krb5kdc[26641](info): set up 2 sockets
Aug 06 06:39:54 dist-test-slave-3kk6 krb5kdc[26641](info): commencing operation
krb5kdc: starting...
W20260806 06:39:55.359617 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:39:55 dist-test-slave-3kk6 krb5kdc[26641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:39:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:39:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:39:55.486460 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44057
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:55.571514 26661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:55.571805 26661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:55.571851 26661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:55.573024 26661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:55.573097 26661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:55.573124 26661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:55.573158 26661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:55.573179 26661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:55.574769 26661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44057
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:55.575048 26661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:55.575290 26661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:55.580404 26667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:55.580400 26671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:55.580619 26661 server_base.cc:1034] running on GCE node
W20260806 06:39:55.580405 26669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:55.580906 26661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:55.581238 26661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:55.582422 26661 hybrid_clock.cc:648] HybridClock initialized: now 1785998395582403 us; error 46 us; skew 500 ppm
Aug 06 06:39:55 dist-test-slave-3kk6 krb5kdc[26641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998395, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:55.589916 26661 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:39:55.590458 26661 webserver.cc:460] Webserver started at http://127.0.0.1:37595/ using document root <none> and password file <none>
I20260806 06:39:55.590766 26661 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:55.590842 26661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:55.591015 26661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:55.592168 26661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "b7e8d88560174842bcac97848a239216"
format_stamp: "Formatted at 2026-08-06 06:39:55 on dist-test-slave-3kk6"
server_key: "56050fba12ce83693344fac0e9cf83fb"
server_key_iv: "1436454db934cd603ba77b1aee933efd"
server_key_version: "encryptionkey@0"
I20260806 06:39:55.592568 26661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "b7e8d88560174842bcac97848a239216"
format_stamp: "Formatted at 2026-08-06 06:39:55 on dist-test-slave-3kk6"
server_key: "56050fba12ce83693344fac0e9cf83fb"
server_key_iv: "1436454db934cd603ba77b1aee933efd"
server_key_version: "encryptionkey@0"
I20260806 06:39:55.594007 26661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:39:55.594887 26680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:55.595073 26661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:55.595198 26661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "b7e8d88560174842bcac97848a239216"
format_stamp: "Formatted at 2026-08-06 06:39:55 on dist-test-slave-3kk6"
server_key: "56050fba12ce83693344fac0e9cf83fb"
server_key_iv: "1436454db934cd603ba77b1aee933efd"
server_key_version: "encryptionkey@0"
I20260806 06:39:55.595281 26661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:55.598749 26661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:55.600865 26661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:55.601042 26661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:55.605468 26661 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44057
I20260806 06:39:55.605507 26753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44057 every 8 connection(s)
I20260806 06:39:55.605882 26661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:55.606956 26754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:55.609289 26754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216: Bootstrap starting.
I20260806 06:39:55.609958 26754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:55.610252 26754 log.cc:824] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:55.610991 26754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216: No bootstrap required, opened a new log
I20260806 06:39:55.612046 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 26661
I20260806 06:39:55.612208 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:39:55.612407 30266 external_mini_cluster.cc:1428] Setting key 7c2f259038e4a943196ed0eac3e5a9d1
I20260806 06:39:55.612927 26754 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e8d88560174842bcac97848a239216" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44057 } }
I20260806 06:39:55.613132 26754 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:55.613245 26754 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e8d88560174842bcac97848a239216, State: Initialized, Role: FOLLOWER
I20260806 06:39:55.613413 26754 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e8d88560174842bcac97848a239216" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44057 } }
I20260806 06:39:55.613502 26754 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:55.613535 26754 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:55.613595 26754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:55.614423 26754 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e8d88560174842bcac97848a239216" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44057 } }
I20260806 06:39:55.614539 26754 leader_election.cc:302] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7e8d88560174842bcac97848a239216; no voters:
I20260806 06:39:55.614724 26754 leader_election.cc:288] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:39:55.614765 26762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:55.614933 26762 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [term 1 LEADER]: Becoming Leader. State: Replica: b7e8d88560174842bcac97848a239216, State: Running, Role: LEADER
I20260806 06:39:55.614959 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:55.613046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38537 (local address 127.0.0.1:44057)
0806 06:39:55.613270 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:55.613280 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:55.613316 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:39:55.613435 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:55.613443 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:55.613473 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:55.613627 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:55.614338 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:55.614342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:55.614342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:55.614347 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:55.614390 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:55.614405 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:55.614487 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:55.614577 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:39:55.614623 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":119,"threads_started":1}
I20260806 06:39:55.614984 26754 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:55.615096 26762 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e8d88560174842bcac97848a239216" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44057 } }
I20260806 06:39:55.615774 26765 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7e8d88560174842bcac97848a239216" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e8d88560174842bcac97848a239216" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44057 } } }
I20260806 06:39:55.616099 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:39:55.616238 26765 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:55.616147 26764 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7e8d88560174842bcac97848a239216. Latest consensus state: current_term: 1 leader_uuid: "b7e8d88560174842bcac97848a239216" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e8d88560174842bcac97848a239216" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44057 } } }
I20260806 06:39:55.616564 26764 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:55.616647 26776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:55.617230 26776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:39:55.619083 26776 catalog_manager.cc:1347] Generated new cluster ID: 979b0f11b8a34ef7bee0b406ac56d8e4
I20260806 06:39:55.619144 26776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:39:55.619227 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:55.617854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47877 (local address 127.0.0.1:44057)
0806 06:39:55.617889 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:55.617892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:55.618055 (+ 163us) server_negotiation.cc:407] Connection header received
0806 06:39:55.618154 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:55.618155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:55.618161 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:55.618189 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:55.618937 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:55.618938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:55.618939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:55.618943 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:55.618958 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:55.618963 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:55.619028 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:55.619150 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 06:39:55.619167 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:39:55.627053 26776 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:39:55.627218 26776 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:55.634563 26776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7e8d88560174842bcac97848a239216: Generated new TSK 0
I20260806 06:39:55.634809 26776 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:39:55.639065 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 26661
2026-08-06T06:39:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:39:55 dist-test-slave-3kk6 krb5kdc[26812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:39:55 dist-test-slave-3kk6 krb5kdc[26812](info): set up 2 sockets
Aug 06 06:39:55 dist-test-slave-3kk6 krb5kdc[26812](info): commencing operation
krb5kdc: starting...
W20260806 06:39:56.855068 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:39:56 dist-test-slave-3kk6 krb5kdc[26812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:39:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:39:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:39:56.987303 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35129
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:57.074713 26828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:57.074950 26828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:57.074975 26828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:57.076161 26828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:57.076210 26828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:57.076224 26828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:57.076237 26828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:57.076295 26828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:57.077803 26828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:57.078076 26828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:57.078329 26828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:57.083524 26837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:57.083523 26836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:57.083523 26839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:57.083847 26828 server_base.cc:1034] running on GCE node
I20260806 06:39:57.084131 26828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:57.084453 26828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:57.085649 26828 hybrid_clock.cc:648] HybridClock initialized: now 1785998397085624 us; error 41 us; skew 500 ppm
Aug 06 06:39:57 dist-test-slave-3kk6 krb5kdc[26812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998397, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:57.088590 26828 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:39:57.089087 26828 webserver.cc:460] Webserver started at http://127.0.0.1:39293/ using document root <none> and password file <none>
I20260806 06:39:57.089342 26828 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:57.089416 26828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:57.089569 26828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:57.090771 26828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "69294a902151473ab082f347b15c866a"
format_stamp: "Formatted at 2026-08-06 06:39:57 on dist-test-slave-3kk6"
server_key: "fc97920d80bb62e0343110adad2db1f1"
server_key_iv: "867aeafe5fcfd6c125f752e4a9a92930"
server_key_version: "encryptionkey@0"
I20260806 06:39:57.091195 26828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "69294a902151473ab082f347b15c866a"
format_stamp: "Formatted at 2026-08-06 06:39:57 on dist-test-slave-3kk6"
server_key: "fc97920d80bb62e0343110adad2db1f1"
server_key_iv: "867aeafe5fcfd6c125f752e4a9a92930"
server_key_version: "encryptionkey@0"
I20260806 06:39:57.092674 26828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:39:57.093503 26853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:57.093752 26828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:57.093875 26828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "69294a902151473ab082f347b15c866a"
format_stamp: "Formatted at 2026-08-06 06:39:57 on dist-test-slave-3kk6"
server_key: "fc97920d80bb62e0343110adad2db1f1"
server_key_iv: "867aeafe5fcfd6c125f752e4a9a92930"
server_key_version: "encryptionkey@0"
I20260806 06:39:57.093957 26828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:57.113869 26828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:57.116189 26828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:57.116393 26828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:57.120777 26828 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35129
I20260806 06:39:57.120826 26934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35129 every 8 connection(s)
I20260806 06:39:57.121222 26828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:57.122176 26935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:57.123286 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 26828
I20260806 06:39:57.123427 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:39:57.123623 30266 external_mini_cluster.cc:1428] Setting key d6bdb827aa9148ca1e1b3a8787079bdb
I20260806 06:39:57.125209 26935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a: Bootstrap starting.
I20260806 06:39:57.125835 26935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:57.126212 26935 log.cc:824] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:57.127028 26935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a: No bootstrap required, opened a new log
I20260806 06:39:57.129364 26935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69294a902151473ab082f347b15c866a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35129 } }
I20260806 06:39:57.129547 26935 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:57.129580 26935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69294a902151473ab082f347b15c866a, State: Initialized, Role: FOLLOWER
I20260806 06:39:57.129721 26935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69294a902151473ab082f347b15c866a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35129 } }
I20260806 06:39:57.129802 26935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:57.129825 26935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:57.129892 26935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:57.130509 26935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69294a902151473ab082f347b15c866a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35129 } }
I20260806 06:39:57.130605 26935 leader_election.cc:302] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69294a902151473ab082f347b15c866a; no voters:
I20260806 06:39:57.130800 26935 leader_election.cc:288] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:39:57.130860 26942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:57.131024 26942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [term 1 LEADER]: Becoming Leader. State: Replica: 69294a902151473ab082f347b15c866a, State: Running, Role: LEADER
I20260806 06:39:57.131079 26935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:57.131258 26942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69294a902151473ab082f347b15c866a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35129 } }
Aug 06 06:39:57 dist-test-slave-3kk6 krb5kdc[26812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:39:57.132325 26943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69294a902151473ab082f347b15c866a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69294a902151473ab082f347b15c866a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35129 } } }
I20260806 06:39:57.132426 26943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:57.132688 26959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:57.132840 26944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69294a902151473ab082f347b15c866a. Latest consensus state: current_term: 1 leader_uuid: "69294a902151473ab082f347b15c866a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69294a902151473ab082f347b15c866a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35129 } } }
I20260806 06:39:57.133256 26944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:57.133685 26959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:39:57.135902 26959 catalog_manager.cc:1347] Generated new cluster ID: 764e5c9332be43f4b17c2142056268ea
I20260806 06:39:57.136019 26959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:39:57.136083 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:57.124219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48307 (local address 127.0.0.1:35129)
0806 06:39:57.124425 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:57.124436 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:57.124472 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:39:57.125009 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:57.125019 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:57.125081 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:57.125240 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:57.125960 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:57.127324 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:57.128076 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:57.128757 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:57.128897 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:57.132137 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:57.132140 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:57.132141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:57.132165 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:57.134108 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:57.134554 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:57.134557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:57.134558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:57.134708 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:57.134978 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:57.134979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:57.134979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:57.135308 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:57.135462 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:57.135708 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 06:39:57.135820 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
I20260806 06:39:57.137049 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:39:57.145356 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:57.138511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58219 (local address 127.0.0.1:35129)
0806 06:39:57.138551 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:57.138553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:57.138821 (+ 268us) server_negotiation.cc:407] Connection header received
0806 06:39:57.138886 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:57.138887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:57.138897 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:57.138953 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:57.139491 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:57.140260 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:57.140978 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:57.141227 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:57.141308 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:57.142580 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:57.142581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:57.142581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:39:57.142588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:57.144143 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:57.144534 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:57.144535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:57.144535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:57.144603 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:39:57.144819 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:39:57.144820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:39:57.144820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:39:57.144940 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:57.145047 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:57.145156 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 06:39:57.145240 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:39:57.159554 26959 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:39:57.160333 26959 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:57.171021 26959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69294a902151473ab082f347b15c866a: Generated new TSK 0
I20260806 06:39:57.171291 26959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:39:57.174763 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 26828
2026-08-06T06:39:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:39:57 dist-test-slave-3kk6 krb5kdc[26980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:39:57 dist-test-slave-3kk6 krb5kdc[26980](info): set up 2 sockets
Aug 06 06:39:57 dist-test-slave-3kk6 krb5kdc[26980](info): commencing operation
krb5kdc: starting...
W20260806 06:39:58.409884 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:39:58 dist-test-slave-3kk6 krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:39:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:39:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:39:58.542292 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:39:58.637998 27002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:39:58.638316 27002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:39:58.638365 27002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:39:58.639804 27002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:39:58.639899 27002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:39:58.639932 27002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:39:58.639966 27002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:39:58.640002 27002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:39:58.641734 27002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:39:58.642077 27002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:39:58.642361 27002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:39:58.647650 27011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:58.647704 27009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:39:58.647785 27008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:39:58.647969 27002 server_base.cc:1034] running on GCE node
I20260806 06:39:58.648257 27002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:39:58.648604 27002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:39:58.649807 27002 hybrid_clock.cc:648] HybridClock initialized: now 1785998398649783 us; error 44 us; skew 500 ppm
Aug 06 06:39:58 dist-test-slave-3kk6 krb5kdc[26980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998398, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:39:58.652541 27002 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:39:58.653025 27002 webserver.cc:460] Webserver started at http://127.0.0.1:36191/ using document root <none> and password file <none>
I20260806 06:39:58.653286 27002 fs_manager.cc:359] Metadata directory not provided
I20260806 06:39:58.653360 27002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:39:58.653525 27002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:39:58.654716 27002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "46e5e90679bb4ca4b33c01c8f0338872"
format_stamp: "Formatted at 2026-08-06 06:39:58 on dist-test-slave-3kk6"
server_key: "e4554f27e7153a3b66f534a2370e5df1"
server_key_iv: "4746171ac4c5487a7cd8555a622a1596"
server_key_version: "encryptionkey@0"
I20260806 06:39:58.655128 27002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "46e5e90679bb4ca4b33c01c8f0338872"
format_stamp: "Formatted at 2026-08-06 06:39:58 on dist-test-slave-3kk6"
server_key: "e4554f27e7153a3b66f534a2370e5df1"
server_key_iv: "4746171ac4c5487a7cd8555a622a1596"
server_key_version: "encryptionkey@0"
I20260806 06:39:58.656579 27002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:39:58.657477 27019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:39:58.657680 27002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:39:58.657828 27002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "46e5e90679bb4ca4b33c01c8f0338872"
format_stamp: "Formatted at 2026-08-06 06:39:58 on dist-test-slave-3kk6"
server_key: "e4554f27e7153a3b66f534a2370e5df1"
server_key_iv: "4746171ac4c5487a7cd8555a622a1596"
server_key_version: "encryptionkey@0"
I20260806 06:39:58.657917 27002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:39:58.679520 27002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:39:58.681758 27002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:39:58.681947 27002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:39:58.686082 27002 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33135
I20260806 06:39:58.686136 27095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33135 every 8 connection(s)
I20260806 06:39:58.686543 27002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:39:58.687511 27096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:39:58.689015 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 27002
I20260806 06:39:58.689146 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:39:58.689352 30266 external_mini_cluster.cc:1428] Setting key ce7f650dcd3f10114cdf1e881d2477db
I20260806 06:39:58.690295 27096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872: Bootstrap starting.
I20260806 06:39:58.691044 27096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872: Neither blocks nor log segments found. Creating new log.
I20260806 06:39:58.691385 27096 log.cc:824] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872: Log is configured to *not* fsync() on all Append() calls
I20260806 06:39:58.692145 27096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872: No bootstrap required, opened a new log
I20260806 06:39:58.694550 27096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e5e90679bb4ca4b33c01c8f0338872" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33135 } }
I20260806 06:39:58.694777 27096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:39:58.694806 27096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46e5e90679bb4ca4b33c01c8f0338872, State: Initialized, Role: FOLLOWER
I20260806 06:39:58.695019 27096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e5e90679bb4ca4b33c01c8f0338872" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33135 } }
I20260806 06:39:58.695122 27096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:39:58.695191 27096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:39:58.695259 27096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:39:58.695340 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:58.689932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52633 (local address 127.0.0.1:33135)
0806 06:39:58.690120 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:58.690128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:58.690154 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:39:58.690610 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:58.690618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:58.690724 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:58.690841 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:58.691087 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:58.692072 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:58.693173 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:58.693600 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:58.693692 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:58.694465 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:58.694470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:58.694470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:58.694741 (+ 271us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:58.694795 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:58.694811 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:58.694925 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:58.695009 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:39:58.695079 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260806 06:39:58.696013 27096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e5e90679bb4ca4b33c01c8f0338872" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33135 } }
I20260806 06:39:58.696137 27096 leader_election.cc:302] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46e5e90679bb4ca4b33c01c8f0338872; no voters:
I20260806 06:39:58.696318 27096 leader_election.cc:288] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:39:58.696367 27101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:39:58.696398 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:39:58.696555 27101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [term 1 LEADER]: Becoming Leader. State: Replica: 46e5e90679bb4ca4b33c01c8f0338872, State: Running, Role: LEADER
I20260806 06:39:58.696606 27096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:39:58.696717 27101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e5e90679bb4ca4b33c01c8f0338872" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33135 } }
I20260806 06:39:58.697160 27102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46e5e90679bb4ca4b33c01c8f0338872" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e5e90679bb4ca4b33c01c8f0338872" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33135 } } }
I20260806 06:39:58.697242 27102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [sys.catalog]: This master's current role is: LEADER
I20260806 06:39:58.697463 27103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46e5e90679bb4ca4b33c01c8f0338872. Latest consensus state: current_term: 1 leader_uuid: "46e5e90679bb4ca4b33c01c8f0338872" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e5e90679bb4ca4b33c01c8f0338872" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33135 } } }
I20260806 06:39:58.697779 27103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872 [sys.catalog]: This master's current role is: LEADER
W20260806 06:39:58.698746 27136 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:39:58.698824 27136 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:39:58.698884 27115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:39:58.699100 27115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:39:58.700572 27115 catalog_manager.cc:1347] Generated new cluster ID: 4ff0379307844ea8af5372e21435d656
I20260806 06:39:58.700623 27115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:39:58.701711 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:39:58.698097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41123 (local address 127.0.0.1:33135)
0806 06:39:58.698132 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:39:58.698133 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:39:58.698362 (+ 229us) server_negotiation.cc:407] Connection header received
0806 06:39:58.698452 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:39:58.698453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:39:58.698461 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:39:58.698503 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:39:58.698717 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:58.699393 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:58.700402 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:39:58.700713 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:39:58.700781 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:39:58.701455 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:39:58.701456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:39:58.701457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:39:58.701461 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:39:58.701476 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:39:58.701480 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:39:58.701526 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:39:58.701589 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 06:39:58.701614 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:39:58.707424 27115 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:39:58.708006 27115 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:39:58.712026 27115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46e5e90679bb4ca4b33c01c8f0338872: Generated new TSK 0
I20260806 06:39:58.712230 27115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:39:58.716545 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 27002
2026-08-06T06:39:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:39:58 dist-test-slave-3kk6 krb5kdc[27141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:39:58 dist-test-slave-3kk6 krb5kdc[27141](info): set up 2 sockets
Aug 06 06:39:58 dist-test-slave-3kk6 krb5kdc[27141](info): commencing operation
krb5kdc: starting...
W20260806 06:39:59.949163 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:00 dist-test-slave-3kk6 krb5kdc[27141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:40:00.082170 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33865
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:00.175639 27159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:00.175915 27159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:00.175958 27159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:00.177167 27159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:00.177241 27159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:00.177277 27159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:00.177306 27159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:00.177332 27159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:00.179064 27159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:00.179368 27159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:00.179621 27159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:00.185020 27166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:00.185063 27169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:00.185170 27159 server_base.cc:1034] running on GCE node
W20260806 06:40:00.185040 27165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:00.185513 27159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:00.185832 27159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:00.187037 27159 hybrid_clock.cc:648] HybridClock initialized: now 1785998400187022 us; error 47 us; skew 500 ppm
Aug 06 06:40:00 dist-test-slave-3kk6 krb5kdc[27141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998400, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:00.190029 27159 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:40:00.190529 27159 webserver.cc:460] Webserver started at http://127.0.0.1:34991/ using document root <none> and password file <none>
I20260806 06:40:00.190937 27159 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:00.191017 27159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:00.191190 27159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:00.192368 27159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "b4db38f40a3140dbac48b6d4ce76b84c"
format_stamp: "Formatted at 2026-08-06 06:40:00 on dist-test-slave-3kk6"
server_key: "8ae2023799a6f5bc2cacea99cd4a4930"
server_key_iv: "84a9657ebc4f53590d254ccb9f81e0c5"
server_key_version: "encryptionkey@0"
I20260806 06:40:00.192787 27159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "b4db38f40a3140dbac48b6d4ce76b84c"
format_stamp: "Formatted at 2026-08-06 06:40:00 on dist-test-slave-3kk6"
server_key: "8ae2023799a6f5bc2cacea99cd4a4930"
server_key_iv: "84a9657ebc4f53590d254ccb9f81e0c5"
server_key_version: "encryptionkey@0"
I20260806 06:40:00.194303 27159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:40:00.195207 27178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:00.195441 27159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:40:00.195549 27159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "b4db38f40a3140dbac48b6d4ce76b84c"
format_stamp: "Formatted at 2026-08-06 06:40:00 on dist-test-slave-3kk6"
server_key: "8ae2023799a6f5bc2cacea99cd4a4930"
server_key_iv: "84a9657ebc4f53590d254ccb9f81e0c5"
server_key_version: "encryptionkey@0"
I20260806 06:40:00.195636 27159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:00.199373 27159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:00.201715 27159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:00.201897 27159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:00.206447 27159 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33865
I20260806 06:40:00.206501 27262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33865 every 8 connection(s)
I20260806 06:40:00.206977 27159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:00.208214 27263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:00.208314 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 27159
I20260806 06:40:00.208436 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:00.208659 30266 external_mini_cluster.cc:1428] Setting key a0c8281db38cdf960686c0b3e760631a
I20260806 06:40:00.211714 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:00.209417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53399 (local address 127.0.0.1:33865)
0806 06:40:00.209655 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:00.209665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:00.209703 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:40:00.210209 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:00.210219 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:00.210264 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:00.210436 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:00.211214 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:00.211218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:00.211219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:00.211224 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:00.211286 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:00.211303 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:00.211381 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:00.211442 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 06:40:00.211488 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":104,"threads_started":1}
I20260806 06:40:00.212297 27263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c: Bootstrap starting.
I20260806 06:40:00.212767 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:00.213017 27263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:00.213326 27263 log.cc:824] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:00.214248 27263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c: No bootstrap required, opened a new log
I20260806 06:40:00.215770 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:00.214484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53289 (local address 127.0.0.1:33865)
0806 06:40:00.214531 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:00.214533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:00.214675 (+ 142us) server_negotiation.cc:407] Connection header received
0806 06:40:00.214802 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:00.214803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:00.214808 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:00.214884 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:00.215543 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:00.215545 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:00.215545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:00.215549 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:00.215567 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:00.215572 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:00.215643 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:00.215690 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 06:40:00.215704 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:40:00.216746 27263 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4db38f40a3140dbac48b6d4ce76b84c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260806 06:40:00.216909 27263 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:00.216935 27263 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4db38f40a3140dbac48b6d4ce76b84c, State: Initialized, Role: FOLLOWER
I20260806 06:40:00.217028 27263 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4db38f40a3140dbac48b6d4ce76b84c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260806 06:40:00.217092 27263 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:00.217113 27263 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:00.217142 27263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:00.217832 27263 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4db38f40a3140dbac48b6d4ce76b84c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260806 06:40:00.217929 27263 leader_election.cc:302] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4db38f40a3140dbac48b6d4ce76b84c; no voters:
I20260806 06:40:00.218082 27263 leader_election.cc:288] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:00.218164 27275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:00.218327 27263 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:00.218353 27275 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [term 1 LEADER]: Becoming Leader. State: Replica: b4db38f40a3140dbac48b6d4ce76b84c, State: Running, Role: LEADER
I20260806 06:40:00.218477 27275 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4db38f40a3140dbac48b6d4ce76b84c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } }
I20260806 06:40:00.218947 27277 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4db38f40a3140dbac48b6d4ce76b84c. Latest consensus state: current_term: 1 leader_uuid: "b4db38f40a3140dbac48b6d4ce76b84c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4db38f40a3140dbac48b6d4ce76b84c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } } }
I20260806 06:40:00.219084 27277 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:00.219307 27289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:00.218933 27276 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4db38f40a3140dbac48b6d4ce76b84c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4db38f40a3140dbac48b6d4ce76b84c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33865 } } }
I20260806 06:40:00.219522 27276 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:00.219911 27289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:00.221881 27289 catalog_manager.cc:1347] Generated new cluster ID: 5fab374803334520ae36aa3990ce0f3d
I20260806 06:40:00.221941 27289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:40:00.242785 27289 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:00.243013 27289 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:00.250695 27289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4db38f40a3140dbac48b6d4ce76b84c: Generated new TSK 0
I20260806 06:40:00.251017 27289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:00.255456 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 27159
2026-08-06T06:40:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1550 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:00 dist-test-slave-3kk6 krb5kdc[27304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:00 dist-test-slave-3kk6 krb5kdc[27304](info): set up 2 sockets
Aug 06 06:40:00 dist-test-slave-3kk6 krb5kdc[27304](info): commencing operation
krb5kdc: starting...
W20260806 06:40:01.538610 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:01 dist-test-slave-3kk6 krb5kdc[27304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:40:01.673516 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43771
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:01.761627 27320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:01.761891 27320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:01.761920 27320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:01.763206 27320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:01.763273 27320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:01.763289 27320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:01.763303 27320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:01.763316 27320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:01.764999 27320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:01.765323 27320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:01.765602 27320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:01.771066 27330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:01.771091 27327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:01.771085 27326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:01.771425 27320 server_base.cc:1034] running on GCE node
I20260806 06:40:01.771699 27320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:01.772035 27320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:01.773229 27320 hybrid_clock.cc:648] HybridClock initialized: now 1785998401773200 us; error 52 us; skew 500 ppm
Aug 06 06:40:01 dist-test-slave-3kk6 krb5kdc[27304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998401, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:01.780484 27320 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:40:01.781080 27320 webserver.cc:460] Webserver started at http://127.0.0.1:43467/ using document root <none> and password file <none>
I20260806 06:40:01.781374 27320 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:01.781452 27320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:01.781620 27320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:01.783001 27320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "9e7a317465bb41dfa0ea49974f603125"
format_stamp: "Formatted at 2026-08-06 06:40:01 on dist-test-slave-3kk6"
server_key: "c67941d59de5658271d94f13a434a38c"
server_key_iv: "b08d5ceb9b16ab096886667bb7a4183b"
server_key_version: "encryptionkey@0"
I20260806 06:40:01.783411 27320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "9e7a317465bb41dfa0ea49974f603125"
format_stamp: "Formatted at 2026-08-06 06:40:01 on dist-test-slave-3kk6"
server_key: "c67941d59de5658271d94f13a434a38c"
server_key_iv: "b08d5ceb9b16ab096886667bb7a4183b"
server_key_version: "encryptionkey@0"
I20260806 06:40:01.784952 27320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:40:01.785813 27337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:01.786074 27320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:40:01.786208 27320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "9e7a317465bb41dfa0ea49974f603125"
format_stamp: "Formatted at 2026-08-06 06:40:01 on dist-test-slave-3kk6"
server_key: "c67941d59de5658271d94f13a434a38c"
server_key_iv: "b08d5ceb9b16ab096886667bb7a4183b"
server_key_version: "encryptionkey@0"
I20260806 06:40:01.786296 27320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:01.795504 27320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:01.797986 27320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:01.798194 27320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:01.802911 27320 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43771
I20260806 06:40:01.802973 27405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43771 every 8 connection(s)
I20260806 06:40:01.803393 27320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:01.804363 27406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:01.806898 27406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125: Bootstrap starting.
I20260806 06:40:01.807495 27406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:01.807776 27406 log.cc:824] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:01.808530 27406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125: No bootstrap required, opened a new log
I20260806 06:40:01.810000 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 27320
I20260806 06:40:01.810150 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:01.810367 30266 external_mini_cluster.cc:1428] Setting key ec536bffb7cf4fa85bf365398e1e89a6
I20260806 06:40:01.810317 27406 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e7a317465bb41dfa0ea49974f603125" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43771 } }
I20260806 06:40:01.810520 27406 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:01.810577 27406 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e7a317465bb41dfa0ea49974f603125, State: Initialized, Role: FOLLOWER
I20260806 06:40:01.810766 27406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e7a317465bb41dfa0ea49974f603125" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43771 } }
I20260806 06:40:01.810837 27406 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:01.810883 27406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:01.810938 27406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:01.811990 27406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e7a317465bb41dfa0ea49974f603125" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43771 } }
I20260806 06:40:01.812136 27406 leader_election.cc:302] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e7a317465bb41dfa0ea49974f603125; no voters:
I20260806 06:40:01.812335 27406 leader_election.cc:288] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:01.812403 27415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:01.812585 27415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [term 1 LEADER]: Becoming Leader. State: Replica: 9e7a317465bb41dfa0ea49974f603125, State: Running, Role: LEADER
I20260806 06:40:01.812705 27406 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:01.812750 27415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e7a317465bb41dfa0ea49974f603125" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43771 } }
I20260806 06:40:01.813198 27415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e7a317465bb41dfa0ea49974f603125. Latest consensus state: current_term: 1 leader_uuid: "9e7a317465bb41dfa0ea49974f603125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e7a317465bb41dfa0ea49974f603125" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43771 } } }
I20260806 06:40:01.813211 27417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e7a317465bb41dfa0ea49974f603125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e7a317465bb41dfa0ea49974f603125" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43771 } } }
I20260806 06:40:01.813340 27415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:01.813355 27417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:01.813591 27429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:01.814172 27429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:01.815984 27429 catalog_manager.cc:1347] Generated new cluster ID: 5f23aceb094f47bead2d4a8c90339dc1
I20260806 06:40:01.816047 27429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:40:01 dist-test-slave-3kk6 krb5kdc[27304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:40:01.824299 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:01.811088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:49129 (local address 127.0.0.1:43771)
0806 06:40:01.811339 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:01.811351 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:01.811394 (+ 43us) server_negotiation.cc:407] Connection header received
0806 06:40:01.811531 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:01.811538 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:01.811587 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:01.811747 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:01.812619 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:01.814061 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:01.815257 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:01.815821 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:01.815940 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:01.819252 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:01.819257 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:01.819257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:01.819272 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:01.821939 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:01.822538 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:01.822542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:01.822542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:01.822719 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:01.823003 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:01.823004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:01.823004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:01.823341 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:01.823506 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:01.823695 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 06:40:01.823810 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":120,"threads_started":1}
I20260806 06:40:01.825242 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:01.837546 27429 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:01.838287 27429 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:01.843654 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:01.826858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:37637 (local address 127.0.0.1:43771)
0806 06:40:01.826899 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:01.826901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:01.834998 (+ 8097us) server_negotiation.cc:407] Connection header received
0806 06:40:01.835103 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:01.835106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:01.835122 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:01.835210 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:01.836002 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:01.837200 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:01.838288 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:01.838680 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:01.838761 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:01.840414 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:01.840416 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:01.840416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:01.840424 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:01.842252 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:01.842768 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:01.842769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:01.842770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:01.842843 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:01.843096 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:01.843097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:01.843097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:01.843241 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:01.843377 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:01.843475 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 06:40:01.843549 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":16}
I20260806 06:40:01.845453 27429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e7a317465bb41dfa0ea49974f603125: Generated new TSK 0
I20260806 06:40:01.845681 27429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:01.851316 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 27320
W20260806 06:40:01.864619 27302 connection.cc:535] client connection to 127.0.0.1:43771 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43771): Connection reset by peer (error 104)
2026-08-06T06:40:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1584 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:01 dist-test-slave-3kk6 krb5kdc[27452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:01 dist-test-slave-3kk6 krb5kdc[27452](info): set up 2 sockets
Aug 06 06:40:01 dist-test-slave-3kk6 krb5kdc[27452](info): commencing operation
krb5kdc: starting...
W20260806 06:40:03.132454 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:03 dist-test-slave-3kk6 krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:40:03.266013 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38527
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:03.354887 27470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:03.355167 27470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:03.355214 27470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:03.356483 27470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:03.356570 27470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:03.356594 27470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:03.356617 27470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:03.356648 27470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:03.358300 27470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:03.358623 27470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:03.358942 27470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:03.364283 27481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:03.364326 27470 server_base.cc:1034] running on GCE node
W20260806 06:40:03.364245 27477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:03.364245 27478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:03.364799 27470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:03.365186 27470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:03.366380 27470 hybrid_clock.cc:648] HybridClock initialized: now 1785998403366368 us; error 48 us; skew 500 ppm
Aug 06 06:40:03 dist-test-slave-3kk6 krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998403, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:03.369308 27470 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:40:03.369877 27470 webserver.cc:460] Webserver started at http://127.0.0.1:40083/ using document root <none> and password file <none>
I20260806 06:40:03.370131 27470 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:03.370214 27470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:03.370379 27470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:03.371613 27470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "502639eb5a914617b71e40f5174fe6b4"
format_stamp: "Formatted at 2026-08-06 06:40:03 on dist-test-slave-3kk6"
server_key: "00f0882c3245c9e29df09fb0b236085e"
server_key_iv: "35b67781493e2e3369603bbd0b329cd4"
server_key_version: "encryptionkey@0"
I20260806 06:40:03.372035 27470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "502639eb5a914617b71e40f5174fe6b4"
format_stamp: "Formatted at 2026-08-06 06:40:03 on dist-test-slave-3kk6"
server_key: "00f0882c3245c9e29df09fb0b236085e"
server_key_iv: "35b67781493e2e3369603bbd0b329cd4"
server_key_version: "encryptionkey@0"
I20260806 06:40:03.373672 27470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:40:03.374581 27497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:03.374861 27470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:40:03.374980 27470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "502639eb5a914617b71e40f5174fe6b4"
format_stamp: "Formatted at 2026-08-06 06:40:03 on dist-test-slave-3kk6"
server_key: "00f0882c3245c9e29df09fb0b236085e"
server_key_iv: "35b67781493e2e3369603bbd0b329cd4"
server_key_version: "encryptionkey@0"
I20260806 06:40:03.375066 27470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:03.384387 27470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:03.386879 27470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:03.387061 27470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:03.391445 27470 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38527
I20260806 06:40:03.391484 27562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38527 every 8 connection(s)
I20260806 06:40:03.391937 27470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:03.392222 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 27470
I20260806 06:40:03.392371 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:03.392606 30266 external_mini_cluster.cc:1428] Setting key 2adaa206186fe3c8b7dab59a981c2274
I20260806 06:40:03.393118 27563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:03.396796 27563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4: Bootstrap starting.
I20260806 06:40:03.397574 27563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:03.397951 27563 log.cc:824] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:03.398777 27563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4: No bootstrap required, opened a new log
I20260806 06:40:03.399463 27570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:03.393265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:48651 (local address 127.0.0.1:38527)
0806 06:40:03.393513 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:03.393524 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:03.393560 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:40:03.394087 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:03.394097 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:03.394166 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:03.394338 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:03.394576 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:03.396045 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:03.396982 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:03.397638 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:03.397771 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:03.398630 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:03.398635 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:03.398635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:03.398921 (+ 286us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:03.398965 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:03.398983 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:03.399088 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:03.399225 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 06:40:03.399266 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":123,"threads_started":1}
I20260806 06:40:03.400605 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:03.401104 27563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "502639eb5a914617b71e40f5174fe6b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } }
I20260806 06:40:03.401286 27563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:03.401336 27563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 502639eb5a914617b71e40f5174fe6b4, State: Initialized, Role: FOLLOWER
I20260806 06:40:03.401475 27563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "502639eb5a914617b71e40f5174fe6b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } }
I20260806 06:40:03.401562 27563 raft_consensus.cc:388] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:03.401607 27563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:03.401659 27563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:03.402388 27563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "502639eb5a914617b71e40f5174fe6b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } }
I20260806 06:40:03.402563 27563 leader_election.cc:302] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 502639eb5a914617b71e40f5174fe6b4; no voters:
I20260806 06:40:03.402797 27563 leader_election.cc:288] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:03.402848 27577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:03.403043 27577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [term 1 LEADER]: Becoming Leader. State: Replica: 502639eb5a914617b71e40f5174fe6b4, State: Running, Role: LEADER
I20260806 06:40:03.403106 27563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:03.403229 27577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "502639eb5a914617b71e40f5174fe6b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } }
I20260806 06:40:03.403642 27577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 502639eb5a914617b71e40f5174fe6b4. Latest consensus state: current_term: 1 leader_uuid: "502639eb5a914617b71e40f5174fe6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "502639eb5a914617b71e40f5174fe6b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } } }
I20260806 06:40:03.403777 27579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "502639eb5a914617b71e40f5174fe6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "502639eb5a914617b71e40f5174fe6b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } } }
I20260806 06:40:03.403859 27577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:03.403885 27579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:03.404196 27592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:03.404820 27592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:03.406488 27570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:03.402390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:47039 (local address 127.0.0.1:38527)
0806 06:40:03.402465 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:03.402467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:03.402593 (+ 126us) server_negotiation.cc:407] Connection header received
0806 06:40:03.402707 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:03.402708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:03.402722 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:03.402793 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:03.403018 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:03.403813 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:03.404998 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:03.405267 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:03.405331 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:03.406179 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:03.406181 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:03.406181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:03.406186 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:03.406214 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:03.406220 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:03.406293 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:03.406397 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:40:03.406409 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:40:03.407012 27592 catalog_manager.cc:1347] Generated new cluster ID: d17c276bbe034e56934063b093df9d68
I20260806 06:40:03.407066 27592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:40:03.427970 27592 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:03.428870 27592 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:03.435745 27592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 502639eb5a914617b71e40f5174fe6b4: Generated new TSK 0
I20260806 06:40:03.436017 27592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:03.438799 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 27470
W20260806 06:40:03.463282 27450 connection.cc:535] client connection to 127.0.0.1:38527 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38527): Connection reset by peer (error 104)
2026-08-06T06:40:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1598 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:03 dist-test-slave-3kk6 krb5kdc[27610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:03 dist-test-slave-3kk6 krb5kdc[27610](info): set up 2 sockets
Aug 06 06:40:03 dist-test-slave-3kk6 krb5kdc[27610](info): commencing operation
krb5kdc: starting...
W20260806 06:40:04.733093 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:04 dist-test-slave-3kk6 krb5kdc[27610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:40:04.865187 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36665
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:04.953569 27628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:04.953830 27628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:04.953857 27628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:04.955168 27628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:04.955230 27628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:04.955246 27628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:04.955260 27628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:04.955272 27628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:04.956996 27628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:04.957325 27628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:04.957600 27628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:04.963037 27635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:04.963066 27640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:04.963037 27636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:04.963337 27628 server_base.cc:1034] running on GCE node
I20260806 06:40:04.963629 27628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:04.963936 27628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:04.965114 27628 hybrid_clock.cc:648] HybridClock initialized: now 1785998404965097 us; error 33 us; skew 500 ppm
Aug 06 06:40:04 dist-test-slave-3kk6 krb5kdc[27610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998404, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:04.968044 27628 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:40:04.968575 27628 webserver.cc:460] Webserver started at http://127.0.0.1:45275/ using document root <none> and password file <none>
I20260806 06:40:04.968844 27628 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:04.968896 27628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:04.969036 27628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:04.970191 27628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "95d0825b232c4954922e8b4bca1f03dc"
format_stamp: "Formatted at 2026-08-06 06:40:04 on dist-test-slave-3kk6"
server_key: "eac8b817b404594231a31ce2485acd74"
server_key_iv: "64bcfe1d92d62ee016a268b469177c66"
server_key_version: "encryptionkey@0"
I20260806 06:40:04.970590 27628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "95d0825b232c4954922e8b4bca1f03dc"
format_stamp: "Formatted at 2026-08-06 06:40:04 on dist-test-slave-3kk6"
server_key: "eac8b817b404594231a31ce2485acd74"
server_key_iv: "64bcfe1d92d62ee016a268b469177c66"
server_key_version: "encryptionkey@0"
I20260806 06:40:04.972206 27628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:40:04.972985 27650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:04.973259 27628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:40:04.973385 27628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "95d0825b232c4954922e8b4bca1f03dc"
format_stamp: "Formatted at 2026-08-06 06:40:04 on dist-test-slave-3kk6"
server_key: "eac8b817b404594231a31ce2485acd74"
server_key_iv: "64bcfe1d92d62ee016a268b469177c66"
server_key_version: "encryptionkey@0"
I20260806 06:40:04.973493 27628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:04.977314 27628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:04.979823 27628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:04.980005 27628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:04.984501 27628 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36665
I20260806 06:40:04.984557 27711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36665 every 8 connection(s)
I20260806 06:40:04.984980 27628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:04.986166 27712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:04.988840 27712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc: Bootstrap starting.
I20260806 06:40:04.989424 27712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:04.989686 27712 log.cc:824] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:04.990496 27712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc: No bootstrap required, opened a new log
I20260806 06:40:04.991413 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 27628
I20260806 06:40:04.991554 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:04.991781 30266 external_mini_cluster.cc:1428] Setting key c0e2923d9e2e73681b8936c86270e75e
I20260806 06:40:04.992837 27712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d0825b232c4954922e8b4bca1f03dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36665 } }
I20260806 06:40:04.993060 27712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:04.993109 27712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95d0825b232c4954922e8b4bca1f03dc, State: Initialized, Role: FOLLOWER
I20260806 06:40:04.993336 27712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d0825b232c4954922e8b4bca1f03dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36665 } }
I20260806 06:40:04.993468 27712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:04.993539 27712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:04.993600 27712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:04.994270 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:04.992437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:55749 (local address 127.0.0.1:36665)
0806 06:40:04.992630 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:04.992639 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:04.992669 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:40:04.992782 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:04.992788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:04.992813 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:04.992932 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:04.993762 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:04.993765 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:04.993765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:04.993769 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:04.993811 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:04.993822 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:04.993882 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:04.993985 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 06:40:04.994043 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260806 06:40:04.994956 27712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d0825b232c4954922e8b4bca1f03dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36665 } }
I20260806 06:40:04.995088 27712 leader_election.cc:302] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95d0825b232c4954922e8b4bca1f03dc; no voters:
I20260806 06:40:04.995286 27712 leader_election.cc:288] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:04.995281 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:04.995468 27721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:04.995577 27712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:04.995668 27721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [term 1 LEADER]: Becoming Leader. State: Replica: 95d0825b232c4954922e8b4bca1f03dc, State: Running, Role: LEADER
I20260806 06:40:04.995818 27721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d0825b232c4954922e8b4bca1f03dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36665 } }
I20260806 06:40:04.996248 27721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95d0825b232c4954922e8b4bca1f03dc. Latest consensus state: current_term: 1 leader_uuid: "95d0825b232c4954922e8b4bca1f03dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d0825b232c4954922e8b4bca1f03dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36665 } } }
I20260806 06:40:04.996258 27723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95d0825b232c4954922e8b4bca1f03dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d0825b232c4954922e8b4bca1f03dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36665 } } }
I20260806 06:40:04.996430 27723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:04.996618 27721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:04.996833 27735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:04.997382 27735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:04.998965 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:04.997531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:45349 (local address 127.0.0.1:36665)
0806 06:40:04.997568 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:04.997570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:04.997822 (+ 252us) server_negotiation.cc:407] Connection header received
0806 06:40:04.997877 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:04.997878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:04.997885 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:04.997956 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:04.998703 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:04.998704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:04.998704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:04.998709 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:04.998724 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:04.998728 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:04.998790 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:04.998881 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 06:40:04.998898 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:40:05.003327 27735 catalog_manager.cc:1347] Generated new cluster ID: e42ad1f62cba406a813b043e35af3c48
I20260806 06:40:05.003402 27735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:40:05.031143 27735 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:05.031368 27735 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:05.046399 27735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95d0825b232c4954922e8b4bca1f03dc: Generated new TSK 0
I20260806 06:40:05.046684 27735 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:05.054546 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 27628
2026-08-06T06:40:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1602 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:05 dist-test-slave-3kk6 krb5kdc[27755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:05 dist-test-slave-3kk6 krb5kdc[27755](info): set up 2 sockets
Aug 06 06:40:05 dist-test-slave-3kk6 krb5kdc[27755](info): commencing operation
krb5kdc: starting...
W20260806 06:40:06.273205 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:06 dist-test-slave-3kk6 krb5kdc[27755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:40:06.402446 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33971
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:06.487594 27776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:06.487828 27776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:06.487854 27776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:06.489027 27776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:06.489076 27776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:06.489090 27776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:06.489104 27776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:06.489156 27776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:06.490626 27776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33971
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:06.491035 27776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:06.491283 27776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:06.496343 27792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:06.496349 27790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:06.496321 27787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:06.496737 27776 server_base.cc:1034] running on GCE node
I20260806 06:40:06.497004 27776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:06.497344 27776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:06.498535 27776 hybrid_clock.cc:648] HybridClock initialized: now 1785998406498508 us; error 45 us; skew 500 ppm
Aug 06 06:40:06 dist-test-slave-3kk6 krb5kdc[27755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998406, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:06.501204 27776 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:40:06.501713 27776 webserver.cc:460] Webserver started at http://127.0.0.1:42215/ using document root <none> and password file <none>
I20260806 06:40:06.501974 27776 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:06.502022 27776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:06.502180 27776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:06.503388 27776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "1ea4e681efbf40468b69dcccd16667c1"
format_stamp: "Formatted at 2026-08-06 06:40:06 on dist-test-slave-3kk6"
server_key: "ec4dd58d148605f267cd1b36790ebe2d"
server_key_iv: "f536d1133558297fb68c905894c69f54"
server_key_version: "encryptionkey@0"
I20260806 06:40:06.503782 27776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "1ea4e681efbf40468b69dcccd16667c1"
format_stamp: "Formatted at 2026-08-06 06:40:06 on dist-test-slave-3kk6"
server_key: "ec4dd58d148605f267cd1b36790ebe2d"
server_key_iv: "f536d1133558297fb68c905894c69f54"
server_key_version: "encryptionkey@0"
I20260806 06:40:06.505175 27776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:40:06.505992 27801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:06.506202 27776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:40:06.506297 27776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "1ea4e681efbf40468b69dcccd16667c1"
format_stamp: "Formatted at 2026-08-06 06:40:06 on dist-test-slave-3kk6"
server_key: "ec4dd58d148605f267cd1b36790ebe2d"
server_key_iv: "f536d1133558297fb68c905894c69f54"
server_key_version: "encryptionkey@0"
I20260806 06:40:06.506381 27776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:06.521215 27776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:06.523283 27776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:06.523474 27776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:06.527623 27776 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33971
I20260806 06:40:06.527659 27884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33971 every 8 connection(s)
I20260806 06:40:06.528070 27776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:06.528968 27886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:06.531394 27886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1: Bootstrap starting.
I20260806 06:40:06.532039 27886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:06.532352 27886 log.cc:824] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:06.533123 27886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1: No bootstrap required, opened a new log
I20260806 06:40:06.534873 27886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea4e681efbf40468b69dcccd16667c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33971 } }
I20260806 06:40:06.535018 27886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:06.535043 27886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ea4e681efbf40468b69dcccd16667c1, State: Initialized, Role: FOLLOWER
I20260806 06:40:06.535215 27886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea4e681efbf40468b69dcccd16667c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33971 } }
I20260806 06:40:06.535312 27886 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:06.535337 27886 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:06.535389 27886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:06.535970 27886 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea4e681efbf40468b69dcccd16667c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33971 } }
I20260806 06:40:06.536089 27886 leader_election.cc:302] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ea4e681efbf40468b69dcccd16667c1; no voters:
I20260806 06:40:06.536295 27886 leader_election.cc:288] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:06.536362 27890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:06.536571 27886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:06.536592 27890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [term 1 LEADER]: Becoming Leader. State: Replica: 1ea4e681efbf40468b69dcccd16667c1, State: Running, Role: LEADER
I20260806 06:40:06.536736 27890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea4e681efbf40468b69dcccd16667c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33971 } }
I20260806 06:40:06.537352 27892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ea4e681efbf40468b69dcccd16667c1. Latest consensus state: current_term: 1 leader_uuid: "1ea4e681efbf40468b69dcccd16667c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea4e681efbf40468b69dcccd16667c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33971 } } }
I20260806 06:40:06.537461 27892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:06.537662 27891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ea4e681efbf40468b69dcccd16667c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ea4e681efbf40468b69dcccd16667c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33971 } } }
I20260806 06:40:06.537809 27891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:06.537988 27901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:06.538139 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 27776
I20260806 06:40:06.538262 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:06.538475 30266 external_mini_cluster.cc:1428] Setting key c667ffa73eac2fd84de7311c53249407
I20260806 06:40:06.538731 27901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:06.540814 27901 catalog_manager.cc:1347] Generated new cluster ID: 53f33224e7c6478182fa26c8ca5ea7f1
I20260806 06:40:06.540870 27901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:40:06 dist-test-slave-3kk6 krb5kdc[27755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:40:06.551388 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:06.539367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:48353 (local address 127.0.0.1:33971)
0806 06:40:06.539547 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:06.539557 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:06.539594 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:40:06.539718 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:06.539733 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:06.539794 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:06.539944 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:06.540784 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:06.542212 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:06.543363 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:06.543982 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:06.544056 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:06.547398 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:06.547402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:06.547403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:06.547417 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:06.549512 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:06.550024 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:06.550027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:06.550028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:06.550139 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:06.550394 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:06.550395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:06.550395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:06.550728 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:06.550888 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:06.551040 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:40:06.551141 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
I20260806 06:40:06.552485 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:06.561939 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:06.554115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:39415 (local address 127.0.0.1:33971)
0806 06:40:06.554161 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:06.554163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:06.554335 (+ 172us) server_negotiation.cc:407] Connection header received
0806 06:40:06.554384 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:06.554385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:06.554396 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:06.554445 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:06.555186 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:06.555971 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:06.557034 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:06.557364 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:06.557430 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:06.559019 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:06.559021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:06.559021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:06.559028 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:06.560645 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:06.561092 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:06.561093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:06.561093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:06.561149 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:06.561393 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:06.561394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:06.561394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:06.561514 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:06.561620 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:06.561774 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 06:40:06.561840 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:40:06.568920 27901 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:06.569446 27901 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:06.573945 27901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ea4e681efbf40468b69dcccd16667c1: Generated new TSK 0
I20260806 06:40:06.574163 27901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:06.576782 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 27776
W20260806 06:40:06.589752 27753 connection.cc:535] client connection to 127.0.0.1:33971 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33971): Connection reset by peer (error 104)
2026-08-06T06:40:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:06 dist-test-slave-3kk6 krb5kdc[27926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:06 dist-test-slave-3kk6 krb5kdc[27926](info): set up 2 sockets
Aug 06 06:40:06 dist-test-slave-3kk6 krb5kdc[27926](info): commencing operation
krb5kdc: starting...
W20260806 06:40:07.802686 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:07 dist-test-slave-3kk6 krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:40:07.931871 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42333
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:08.017234 27943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:08.017530 27943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:08.017580 27943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:08.018821 27943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:08.018899 27943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:08.018925 27943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:08.018949 27943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:08.018977 27943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:08.020574 27943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42333
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:08.020895 27943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:08.021154 27943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:08.026340 27957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:08.026372 27960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:08.026551 27955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:08.026686 27943 server_base.cc:1034] running on GCE node
I20260806 06:40:08.026959 27943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:08.027314 27943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:08.028481 27943 hybrid_clock.cc:648] HybridClock initialized: now 1785998408028472 us; error 43 us; skew 500 ppm
Aug 06 06:40:08 dist-test-slave-3kk6 krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998408, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:08.031168 27943 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:40:08.031579 27943 webserver.cc:460] Webserver started at http://127.0.0.1:32859/ using document root <none> and password file <none>
I20260806 06:40:08.031788 27943 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:08.031831 27943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:08.031925 27943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:08.032963 27943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "4baf3050f792472d880dca4446886ed7"
format_stamp: "Formatted at 2026-08-06 06:40:08 on dist-test-slave-3kk6"
server_key: "cf47c5067eaab9c72a6f837d3cbab647"
server_key_iv: "01203cc23c0a5e8971bef405007fe19f"
server_key_version: "encryptionkey@0"
I20260806 06:40:08.033293 27943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "4baf3050f792472d880dca4446886ed7"
format_stamp: "Formatted at 2026-08-06 06:40:08 on dist-test-slave-3kk6"
server_key: "cf47c5067eaab9c72a6f837d3cbab647"
server_key_iv: "01203cc23c0a5e8971bef405007fe19f"
server_key_version: "encryptionkey@0"
I20260806 06:40:08.034612 27943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:40:08.035389 27969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:08.035549 27943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:40:08.035650 27943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "4baf3050f792472d880dca4446886ed7"
format_stamp: "Formatted at 2026-08-06 06:40:08 on dist-test-slave-3kk6"
server_key: "cf47c5067eaab9c72a6f837d3cbab647"
server_key_iv: "01203cc23c0a5e8971bef405007fe19f"
server_key_version: "encryptionkey@0"
I20260806 06:40:08.035707 27943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:08.053684 27943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:08.055984 27943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:08.056114 27943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:08.060113 27943 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42333
I20260806 06:40:08.060235 28036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42333 every 8 connection(s)
I20260806 06:40:08.060588 27943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:08.061539 28037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:08.064098 28037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7: Bootstrap starting.
I20260806 06:40:08.064730 28037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:08.065037 28037 log.cc:824] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:08.065809 28037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7: No bootstrap required, opened a new log
I20260806 06:40:08.067577 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 27943
I20260806 06:40:08.067541 28037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4baf3050f792472d880dca4446886ed7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42333 } }
I20260806 06:40:08.067705 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:08.067739 28037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:08.067778 28037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4baf3050f792472d880dca4446886ed7, State: Initialized, Role: FOLLOWER
I20260806 06:40:08.067929 30266 external_mini_cluster.cc:1428] Setting key e56def2c548093ed0045a95716909c6d
I20260806 06:40:08.067940 28037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4baf3050f792472d880dca4446886ed7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42333 } }
I20260806 06:40:08.068033 28037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:08.068074 28037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:08.068133 28037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:08.068810 28037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4baf3050f792472d880dca4446886ed7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42333 } }
I20260806 06:40:08.068905 28037 leader_election.cc:302] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4baf3050f792472d880dca4446886ed7; no voters:
I20260806 06:40:08.069139 28037 leader_election.cc:288] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:08.069185 28047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:08.069410 28047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [term 1 LEADER]: Becoming Leader. State: Replica: 4baf3050f792472d880dca4446886ed7, State: Running, Role: LEADER
I20260806 06:40:08.069425 28037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:08.070216 28047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4baf3050f792472d880dca4446886ed7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42333 } }
I20260806 06:40:08.070812 28052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4baf3050f792472d880dca4446886ed7. Latest consensus state: current_term: 1 leader_uuid: "4baf3050f792472d880dca4446886ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4baf3050f792472d880dca4446886ed7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42333 } } }
I20260806 06:40:08.070977 28049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4baf3050f792472d880dca4446886ed7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4baf3050f792472d880dca4446886ed7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42333 } } }
I20260806 06:40:08.071208 28052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:08.071224 28049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:08.071561 28069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:08.072170 28069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:08.073807 28069 catalog_manager.cc:1347] Generated new cluster ID: cafc63b630ca41638d07ddcfc31d8850
I20260806 06:40:08.073863 28069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:40:08.075278 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:08.068452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:46181 (local address 127.0.0.1:42333)
0806 06:40:08.069130 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:08.069140 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:08.069180 (+ 40us) server_negotiation.cc:407] Connection header received
0806 06:40:08.069328 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:08.069336 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:08.069397 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:08.069823 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:08.069930 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:08.071527 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:08.072672 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:08.073279 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:08.073393 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:08.074506 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:08.074510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:08.074511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:08.074765 (+ 254us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:08.074812 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:08.074827 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:08.074912 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:08.075050 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 06:40:08.075077 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":560,"threads_started":1}
I20260806 06:40:08.076200 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:08.081552 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:08.077909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:56449 (local address 127.0.0.1:42333)
0806 06:40:08.077941 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:08.077943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:08.077969 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:40:08.078048 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:08.078049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:08.078066 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:08.078121 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:08.078295 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:08.079056 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:08.080104 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:08.080409 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:08.080469 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:08.081282 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:08.081284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:08.081284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:08.081289 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:08.081304 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:08.081309 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:08.081357 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:08.081459 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 06:40:08.081471 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 06:40:08.097245 28069 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:08.098174 28069 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:08.106253 28069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4baf3050f792472d880dca4446886ed7: Generated new TSK 0
I20260806 06:40:08.106505 28069 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:08.112602 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 27943
W20260806 06:40:08.126098 27924 connection.cc:535] client connection to 127.0.0.1:42333 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42333): Connection reset by peer (error 104)
2026-08-06T06:40:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:08 dist-test-slave-3kk6 krb5kdc[28085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:08 dist-test-slave-3kk6 krb5kdc[28085](info): set up 2 sockets
Aug 06 06:40:08 dist-test-slave-3kk6 krb5kdc[28085](info): commencing operation
krb5kdc: starting...
W20260806 06:40:09.325050 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:09 dist-test-slave-3kk6 krb5kdc[28085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:40:09.454005 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43831
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:09.542773 28100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:09.543035 28100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:09.543078 28100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:09.544283 28100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:09.544358 28100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:09.544384 28100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:09.544406 28100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:09.544435 28100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:09.545945 28100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43831
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:09.546258 28100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:09.546515 28100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:09.552078 28109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:09.552035 28105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:09.552035 28106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:09.552317 28100 server_base.cc:1034] running on GCE node
I20260806 06:40:09.552683 28100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:09.552987 28100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:09.554188 28100 hybrid_clock.cc:648] HybridClock initialized: now 1785998409554160 us; error 47 us; skew 500 ppm
Aug 06 06:40:09 dist-test-slave-3kk6 krb5kdc[28085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998409, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:09.557148 28100 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:40:09.557698 28100 webserver.cc:460] Webserver started at http://127.0.0.1:39403/ using document root <none> and password file <none>
I20260806 06:40:09.557948 28100 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:09.557996 28100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:09.558187 28100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:09.559433 28100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "dffa01d1f78a4e2b9a6d190d1d91f0d6"
format_stamp: "Formatted at 2026-08-06 06:40:09 on dist-test-slave-3kk6"
server_key: "9b65a38850e084dfb4a5b04aa4bad255"
server_key_iv: "e394b1107e0d3615f6f6f6dc2de6a4fc"
server_key_version: "encryptionkey@0"
I20260806 06:40:09.559834 28100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "dffa01d1f78a4e2b9a6d190d1d91f0d6"
format_stamp: "Formatted at 2026-08-06 06:40:09 on dist-test-slave-3kk6"
server_key: "9b65a38850e084dfb4a5b04aa4bad255"
server_key_iv: "e394b1107e0d3615f6f6f6dc2de6a4fc"
server_key_version: "encryptionkey@0"
I20260806 06:40:09.561290 28100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:40:09.562112 28117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:09.562340 28100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:40:09.562461 28100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "dffa01d1f78a4e2b9a6d190d1d91f0d6"
format_stamp: "Formatted at 2026-08-06 06:40:09 on dist-test-slave-3kk6"
server_key: "9b65a38850e084dfb4a5b04aa4bad255"
server_key_iv: "e394b1107e0d3615f6f6f6dc2de6a4fc"
server_key_version: "encryptionkey@0"
I20260806 06:40:09.562539 28100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:09.566015 28100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:09.568113 28100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:09.568284 28100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:09.572479 28100 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43831
I20260806 06:40:09.572551 28195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43831 every 8 connection(s)
I20260806 06:40:09.572906 28100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:09.574083 28196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:09.576824 28196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6: Bootstrap starting.
I20260806 06:40:09.577536 28196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:09.577869 28196 log.cc:824] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:09.578774 28196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6: No bootstrap required, opened a new log
I20260806 06:40:09.579792 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 28100
I20260806 06:40:09.579959 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:09.580154 30266 external_mini_cluster.cc:1428] Setting key b14f89a27acaaef59e8f9a608e90f87f
I20260806 06:40:09.581049 28196 raft_consensus.cc:348] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffa01d1f78a4e2b9a6d190d1d91f0d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43831 } }
I20260806 06:40:09.581250 28196 raft_consensus.cc:374] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:09.581315 28196 raft_consensus.cc:729] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dffa01d1f78a4e2b9a6d190d1d91f0d6, State: Initialized, Role: FOLLOWER
I20260806 06:40:09.581523 28196 consensus_queue.cc:260] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffa01d1f78a4e2b9a6d190d1d91f0d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43831 } }
I20260806 06:40:09.581626 28196 raft_consensus.cc:388] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:09.581705 28196 raft_consensus.cc:482] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:09.581777 28196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:09.582572 28196 raft_consensus.cc:504] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffa01d1f78a4e2b9a6d190d1d91f0d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43831 } }
I20260806 06:40:09.582643 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:09.580785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:55301 (local address 127.0.0.1:43831)
0806 06:40:09.581026 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:09.581035 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:09.581070 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:40:09.581174 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:09.581191 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:09.581216 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:09.581373 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:09.582225 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:09.582233 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:09.582234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:09.582238 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:09.582273 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:09.582284 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:09.582357 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:09.582432 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 06:40:09.582479 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":126,"threads_started":1}
I20260806 06:40:09.582736 28196 leader_election.cc:302] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dffa01d1f78a4e2b9a6d190d1d91f0d6; no voters:
I20260806 06:40:09.582942 28196 leader_election.cc:288] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:09.582994 28202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:09.583171 28202 raft_consensus.cc:686] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [term 1 LEADER]: Becoming Leader. State: Replica: dffa01d1f78a4e2b9a6d190d1d91f0d6, State: Running, Role: LEADER
I20260806 06:40:09.583304 28196 sys_catalog.cc:564] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:09.583292 28202 consensus_queue.cc:237] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffa01d1f78a4e2b9a6d190d1d91f0d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43831 } }
I20260806 06:40:09.583745 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:09.583935 28203 sys_catalog.cc:455] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dffa01d1f78a4e2b9a6d190d1d91f0d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffa01d1f78a4e2b9a6d190d1d91f0d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43831 } } }
I20260806 06:40:09.584045 28203 sys_catalog.cc:458] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:09.584321 28205 sys_catalog.cc:455] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dffa01d1f78a4e2b9a6d190d1d91f0d6. Latest consensus state: current_term: 1 leader_uuid: "dffa01d1f78a4e2b9a6d190d1d91f0d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffa01d1f78a4e2b9a6d190d1d91f0d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43831 } } }
I20260806 06:40:09.584456 28205 sys_catalog.cc:458] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:09.584646 28215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:09.585289 28215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:09.587600 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:09.585421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:36937 (local address 127.0.0.1:43831)
0806 06:40:09.585465 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:09.585467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:09.586379 (+ 912us) server_negotiation.cc:407] Connection header received
0806 06:40:09.586415 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:09.586416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:09.586440 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:09.586522 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:09.587328 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:09.587329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:09.587330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:09.587335 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:09.587352 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:09.587356 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:09.587423 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:09.587488 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 06:40:09.587505 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:40:09.587663 28215 catalog_manager.cc:1347] Generated new cluster ID: fc403132adc74b8aba97e8bcf24688af
I20260806 06:40:09.587736 28215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:40:09.622032 28215 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:09.622262 28215 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:09.626708 28215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dffa01d1f78a4e2b9a6d190d1d91f0d6: Generated new TSK 0
I20260806 06:40:09.626929 28215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:09.636462 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 28100
2026-08-06T06:40:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1517 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:09 dist-test-slave-3kk6 krb5kdc[28238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:09 dist-test-slave-3kk6 krb5kdc[28238](info): set up 2 sockets
Aug 06 06:40:09 dist-test-slave-3kk6 krb5kdc[28238](info): commencing operation
krb5kdc: starting...
W20260806 06:40:10.863391 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:10 dist-test-slave-3kk6 krb5kdc[28238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:40:10.997480 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:46715
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:33207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:46715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:11.083271 28258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:11.083555 28258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:11.083599 28258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:11.084810 28258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:11.084893 28258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:11.084921 28258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:11.084954 28258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:11.084982 28258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:11.086542 28258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:33207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:46715
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:46715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:11.086886 28258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:11.087141 28258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:11.092393 28265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:11.092410 28267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:11.092455 28258 server_base.cc:1034] running on GCE node
W20260806 06:40:11.092439 28264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:11.092990 28258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:11.093341 28258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:11.094524 28258 hybrid_clock.cc:648] HybridClock initialized: now 1785998411094502 us; error 42 us; skew 500 ppm
Aug 06 06:40:11 dist-test-slave-3kk6 krb5kdc[28238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998411, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:11.097543 28258 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:40:11.098099 28258 webserver.cc:460] Webserver started at http://127.29.142.190:41215/ using document root <none> and password file <none>
I20260806 06:40:11.098346 28258 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:11.098418 28258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:11.098583 28258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:11.099797 28258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "e0db631978a24ddd87a06e45e36a64d6"
format_stamp: "Formatted at 2026-08-06 06:40:11 on dist-test-slave-3kk6"
server_key: "edb0a75e94d5aee8eb18edcc022bf2d7"
server_key_iv: "1216cb78c5aefd268c66b8c82f6b4a45"
server_key_version: "encryptionkey@0"
I20260806 06:40:11.100198 28258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "e0db631978a24ddd87a06e45e36a64d6"
format_stamp: "Formatted at 2026-08-06 06:40:11 on dist-test-slave-3kk6"
server_key: "edb0a75e94d5aee8eb18edcc022bf2d7"
server_key_iv: "1216cb78c5aefd268c66b8c82f6b4a45"
server_key_version: "encryptionkey@0"
I20260806 06:40:11.101686 28258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:40:11.102525 28278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:11.102768 28258 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:40:11.102896 28258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "e0db631978a24ddd87a06e45e36a64d6"
format_stamp: "Formatted at 2026-08-06 06:40:11 on dist-test-slave-3kk6"
server_key: "edb0a75e94d5aee8eb18edcc022bf2d7"
server_key_iv: "1216cb78c5aefd268c66b8c82f6b4a45"
server_key_version: "encryptionkey@0"
I20260806 06:40:11.102988 28258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:11.123625 28258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:11.125959 28258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:11.126147 28258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:11.130795 28258 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:46715
I20260806 06:40:11.130878 28357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:46715 every 8 connection(s)
I20260806 06:40:11.131275 28258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:11.132215 28358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:11.133878 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 28258
I20260806 06:40:11.134016 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:11.134220 30266 external_mini_cluster.cc:1428] Setting key c79a8d74beff84c2c132c7e62801d8fd
I20260806 06:40:11.135200 28358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6: Bootstrap starting.
I20260806 06:40:11.135898 28358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:11.136255 28358 log.cc:824] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:11.137063 28358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6: No bootstrap required, opened a new log
I20260806 06:40:11.139427 28358 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0db631978a24ddd87a06e45e36a64d6" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46715 } }
I20260806 06:40:11.139626 28358 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:11.139659 28358 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0db631978a24ddd87a06e45e36a64d6, State: Initialized, Role: FOLLOWER
I20260806 06:40:11.139840 28358 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0db631978a24ddd87a06e45e36a64d6" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46715 } }
I20260806 06:40:11.139936 28358 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:11.139971 28358 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:11.140022 28358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:11.140873 28358 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0db631978a24ddd87a06e45e36a64d6" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46715 } }
I20260806 06:40:11.140988 28358 leader_election.cc:302] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0db631978a24ddd87a06e45e36a64d6; no voters:
I20260806 06:40:11.141183 28358 leader_election.cc:288] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:11.141245 28363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:11.141474 28358 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:11.141472 28363 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [term 1 LEADER]: Becoming Leader. State: Replica: e0db631978a24ddd87a06e45e36a64d6, State: Running, Role: LEADER
I20260806 06:40:11.141708 28363 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0db631978a24ddd87a06e45e36a64d6" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46715 } }
Aug 06 06:40:11 dist-test-slave-3kk6 krb5kdc[28238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:40:11.142066 28363 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0db631978a24ddd87a06e45e36a64d6. Latest consensus state: current_term: 1 leader_uuid: "e0db631978a24ddd87a06e45e36a64d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0db631978a24ddd87a06e45e36a64d6" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46715 } } }
I20260806 06:40:11.142320 28363 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:11.142246 28364 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0db631978a24ddd87a06e45e36a64d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0db631978a24ddd87a06e45e36a64d6" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 46715 } } }
I20260806 06:40:11.142807 28364 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:11.143010 28375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:11.143543 28375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:11.145354 28375 catalog_manager.cc:1347] Generated new cluster ID: 8a27b03bc991470aa653ab592dfa0c21
I20260806 06:40:11.145412 28375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:40:11.146736 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:11.134860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58465 (local address 127.29.142.190:46715)
0806 06:40:11.135054 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:11.135062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:11.135091 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:40:11.135562 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:11.135570 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:11.135632 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:11.135754 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:11.136649 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:11.137685 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:11.138821 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:11.139221 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:11.139313 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:11.142535 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:11.142539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:11.142539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:11.142552 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:11.144621 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:11.145149 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:11.145153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:11.145153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:11.145261 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:11.145547 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:11.145548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:11.145548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:11.145841 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:11.145972 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:11.146384 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 06:40:11.146482 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260806 06:40:11.147725 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:11.162643 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:11.151026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52473 (local address 127.29.142.190:46715)
0806 06:40:11.151067 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:11.151069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:11.154875 (+ 3806us) server_negotiation.cc:407] Connection header received
0806 06:40:11.154918 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:11.154919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:11.154932 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:11.155010 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:11.155702 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:11.156529 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:11.157534 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:11.157908 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:11.157989 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:11.159544 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:11.159546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:11.159546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:11.159554 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:11.161356 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:11.161777 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:11.161778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:11.161779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:11.161845 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:11.162077 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:11.162078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:11.162079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:11.162221 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:11.162339 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:11.162462 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 06:40:11.162533 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:40:11.171197 28375 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:11.171836 28375 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:11.181973 28375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0db631978a24ddd87a06e45e36a64d6: Generated new TSK 0
I20260806 06:40:11.182220 28375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:11.187554 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 28258
W20260806 06:40:11.200470 28235 connection.cc:535] client connection to 127.29.142.190:46715 recv error: Network error: failed to read from TLS socket (remote: 127.29.142.190:46715): Connection reset by peer (error 104)
2026-08-06T06:40:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1556 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:11 dist-test-slave-3kk6 krb5kdc[28397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:11 dist-test-slave-3kk6 krb5kdc[28397](info): set up 2 sockets
Aug 06 06:40:11 dist-test-slave-3kk6 krb5kdc[28397](info): commencing operation
krb5kdc: starting...
W20260806 06:40:12.426448 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:12 dist-test-slave-3kk6 krb5kdc[28397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:40:12.555905 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:39113
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:39113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:12.641923 28412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:12.642195 28412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:12.642232 28412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:12.643411 28412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:12.643492 28412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:12.643518 28412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:12.643541 28412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:12.643574 28412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:12.645087 28412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:39113
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:39113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:12.645383 28412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:12.645627 28412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:12.650939 28424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:12.651072 28421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:12.650951 28420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:12.650962 28412 server_base.cc:1034] running on GCE node
I20260806 06:40:12.651580 28412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:12.651983 28412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:12.653156 28412 hybrid_clock.cc:648] HybridClock initialized: now 1785998412653154 us; error 43 us; skew 500 ppm
Aug 06 06:40:12 dist-test-slave-3kk6 krb5kdc[28397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998412, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:12.660600 28412 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:40:12.661150 28412 webserver.cc:460] Webserver started at http://127.29.142.190:35611/ using document root <none> and password file <none>
I20260806 06:40:12.661355 28412 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:12.661398 28412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:12.661497 28412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:12.662714 28412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "5af340b71bae4433bf8695d8060022d8"
format_stamp: "Formatted at 2026-08-06 06:40:12 on dist-test-slave-3kk6"
server_key: "78daf40e1c9ad985008be54109fc383f"
server_key_iv: "88250e393c730bb4cdcee64f1bffda3c"
server_key_version: "encryptionkey@0"
I20260806 06:40:12.663131 28412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "5af340b71bae4433bf8695d8060022d8"
format_stamp: "Formatted at 2026-08-06 06:40:12 on dist-test-slave-3kk6"
server_key: "78daf40e1c9ad985008be54109fc383f"
server_key_iv: "88250e393c730bb4cdcee64f1bffda3c"
server_key_version: "encryptionkey@0"
I20260806 06:40:12.664601 28412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:40:12.665482 28436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:12.665717 28412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:40:12.665828 28412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "5af340b71bae4433bf8695d8060022d8"
format_stamp: "Formatted at 2026-08-06 06:40:12 on dist-test-slave-3kk6"
server_key: "78daf40e1c9ad985008be54109fc383f"
server_key_iv: "88250e393c730bb4cdcee64f1bffda3c"
server_key_version: "encryptionkey@0"
I20260806 06:40:12.665891 28412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:12.682760 28412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:12.685418 28412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:12.685552 28412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:12.689836 28412 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:39113
I20260806 06:40:12.689918 28505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:39113 every 8 connection(s)
I20260806 06:40:12.690343 28412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:12.691411 28506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:12.692193 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 28412
I20260806 06:40:12.692333 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:12.692523 30266 external_mini_cluster.cc:1428] Setting key 52f0de2436b0f3af2aa1cf6b23d61215
I20260806 06:40:12.695055 28506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8: Bootstrap starting.
I20260806 06:40:12.695811 28506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:12.696146 28506 log.cc:824] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:12.696983 28506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8: No bootstrap required, opened a new log
I20260806 06:40:12.699120 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:12.693183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43399 (local address 127.29.142.190:39113)
0806 06:40:12.693429 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:12.693441 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:12.693479 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:40:12.694071 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:12.694082 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:12.694153 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:12.694331 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:12.694514 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:12.695890 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:12.696753 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:12.697371 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:12.697494 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:12.698294 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:12.698300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:12.698301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:12.698536 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:12.698579 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:12.698596 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:12.698749 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:12.698841 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:40:12.698876 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":115,"threads_started":1}
I20260806 06:40:12.699783 28506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af340b71bae4433bf8695d8060022d8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39113 } }
I20260806 06:40:12.699980 28506 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:12.700026 28506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af340b71bae4433bf8695d8060022d8, State: Initialized, Role: FOLLOWER
I20260806 06:40:12.700160 28506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af340b71bae4433bf8695d8060022d8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39113 } }
I20260806 06:40:12.700255 28506 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:12.700289 28506 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:12.700335 28506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:12.700459 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:12.701086 28506 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af340b71bae4433bf8695d8060022d8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39113 } }
I20260806 06:40:12.701186 28506 leader_election.cc:302] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5af340b71bae4433bf8695d8060022d8; no voters:
I20260806 06:40:12.701402 28506 leader_election.cc:288] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:12.701458 28511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:12.701640 28511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [term 1 LEADER]: Becoming Leader. State: Replica: 5af340b71bae4433bf8695d8060022d8, State: Running, Role: LEADER
I20260806 06:40:12.701706 28506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:12.701784 28511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af340b71bae4433bf8695d8060022d8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39113 } }
I20260806 06:40:12.702143 28511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af340b71bae4433bf8695d8060022d8. Latest consensus state: current_term: 1 leader_uuid: "5af340b71bae4433bf8695d8060022d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af340b71bae4433bf8695d8060022d8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39113 } } }
I20260806 06:40:12.702180 28513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5af340b71bae4433bf8695d8060022d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af340b71bae4433bf8695d8060022d8" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 39113 } } }
I20260806 06:40:12.702242 28511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:12.702301 28513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:12.702883 28522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:12.703466 28522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:12.705317 28522 catalog_manager.cc:1347] Generated new cluster ID: 7e35bc812cde48018a22754c99a98cb3
I20260806 06:40:12.705400 28522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:40:12.713028 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:12.702695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40817 (local address 127.29.142.190:39113)
0806 06:40:12.703340 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:12.703343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:12.703652 (+ 309us) server_negotiation.cc:407] Connection header received
0806 06:40:12.703776 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:12.703777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:12.703791 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:12.703858 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:12.704018 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:12.704757 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:12.711373 (+ 6616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:12.711768 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:12.711870 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:12.712743 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:12.712745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:12.712745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:12.712752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:12.712770 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:12.712775 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:12.712844 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:12.712920 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:40:12.712934 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623}
I20260806 06:40:12.726240 28522 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:12.726930 28522 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:12.729070 28522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5af340b71bae4433bf8695d8060022d8: Generated new TSK 0
I20260806 06:40:12.729281 28522 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:12.734452 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 28412
W20260806 06:40:12.749388 28394 connection.cc:535] client connection to 127.29.142.190:39113 recv error: Network error: failed to read from TLS socket (remote: 127.29.142.190:39113): Connection reset by peer (error 104)
2026-08-06T06:40:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:12 dist-test-slave-3kk6 krb5kdc[28544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:12 dist-test-slave-3kk6 krb5kdc[28544](info): set up 2 sockets
Aug 06 06:40:12 dist-test-slave-3kk6 krb5kdc[28544](info): commencing operation
krb5kdc: starting...
W20260806 06:40:13.956626 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:14 dist-test-slave-3kk6 krb5kdc[28544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:40:14.085799 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:33445
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:38595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:33445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:14.174437 28566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:14.174769 28566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:14.174823 28566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:14.176081 28566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:14.176168 28566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:14.176199 28566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:14.176229 28566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:14.176258 28566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:14.177845 28566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:38595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:33445
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:33445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:14.178157 28566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:14.178417 28566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:14.183614 28573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:14.183614 28571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:14.183632 28575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:14.183897 28566 server_base.cc:1034] running on GCE node
I20260806 06:40:14.184145 28566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:14.184489 28566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:14.185653 28566 hybrid_clock.cc:648] HybridClock initialized: now 1785998414185631 us; error 47 us; skew 500 ppm
Aug 06 06:40:14 dist-test-slave-3kk6 krb5kdc[28544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998414, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:14.188562 28566 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:40:14.189106 28566 webserver.cc:460] Webserver started at http://127.29.142.190:40867/ using document root <none> and password file <none>
I20260806 06:40:14.189383 28566 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:14.189459 28566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:14.189622 28566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:14.190845 28566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "a35c98ba02cf40b09f9c49919348a968"
format_stamp: "Formatted at 2026-08-06 06:40:14 on dist-test-slave-3kk6"
server_key: "a7d7cae1ea453ce66690b3f843b59e35"
server_key_iv: "b6cd369ec01075242c84bdd9b2d0cfb0"
server_key_version: "encryptionkey@0"
I20260806 06:40:14.191241 28566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "a35c98ba02cf40b09f9c49919348a968"
format_stamp: "Formatted at 2026-08-06 06:40:14 on dist-test-slave-3kk6"
server_key: "a7d7cae1ea453ce66690b3f843b59e35"
server_key_iv: "b6cd369ec01075242c84bdd9b2d0cfb0"
server_key_version: "encryptionkey@0"
I20260806 06:40:14.192713 28566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:40:14.193521 28585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:14.195026 28566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 06:40:14.195147 28566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "a35c98ba02cf40b09f9c49919348a968"
format_stamp: "Formatted at 2026-08-06 06:40:14 on dist-test-slave-3kk6"
server_key: "a7d7cae1ea453ce66690b3f843b59e35"
server_key_iv: "b6cd369ec01075242c84bdd9b2d0cfb0"
server_key_version: "encryptionkey@0"
I20260806 06:40:14.195245 28566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:14.198817 28566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:14.201175 28566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:14.201383 28566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:14.205888 28566 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:33445
I20260806 06:40:14.205925 28652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:33445 every 8 connection(s)
I20260806 06:40:14.206329 28566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:14.207507 28653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:14.209877 28653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968: Bootstrap starting.
I20260806 06:40:14.210456 28653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:14.210793 28653 log.cc:824] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:14.211598 28653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968: No bootstrap required, opened a new log
I20260806 06:40:14.211866 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 28566
I20260806 06:40:14.211988 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:14.212188 30266 external_mini_cluster.cc:1428] Setting key 8dfde0cbc06f16cc4cba99d2699fb41f
I20260806 06:40:14.214201 28653 raft_consensus.cc:348] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35c98ba02cf40b09f9c49919348a968" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33445 } }
I20260806 06:40:14.214403 28653 raft_consensus.cc:374] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:14.214479 28653 raft_consensus.cc:729] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35c98ba02cf40b09f9c49919348a968, State: Initialized, Role: FOLLOWER
I20260806 06:40:14.214682 28653 consensus_queue.cc:260] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35c98ba02cf40b09f9c49919348a968" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33445 } }
I20260806 06:40:14.215188 28653 raft_consensus.cc:388] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:14.215250 28653 raft_consensus.cc:482] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:14.215305 28657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:14.212812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45825 (local address 127.29.142.190:33445)
0806 06:40:14.213035 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:14.213045 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:14.213083 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:40:14.213707 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:14.213717 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:14.213754 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:14.213928 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:14.214725 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:14.214730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:14.214730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:14.214735 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:14.214807 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:14.214823 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:14.214900 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:14.215004 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 06:40:14.215040 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":97,"threads_started":1}
I20260806 06:40:14.215306 28653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:14.216243 28653 raft_consensus.cc:504] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35c98ba02cf40b09f9c49919348a968" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33445 } }
I20260806 06:40:14.216368 28653 leader_election.cc:302] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a35c98ba02cf40b09f9c49919348a968; no voters:
I20260806 06:40:14.216389 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:14.216594 28653 leader_election.cc:288] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:14.216619 28659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:14.216743 28659 raft_consensus.cc:686] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [term 1 LEADER]: Becoming Leader. State: Replica: a35c98ba02cf40b09f9c49919348a968, State: Running, Role: LEADER
I20260806 06:40:14.216895 28653 sys_catalog.cc:564] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:14.216921 28659 consensus_queue.cc:237] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35c98ba02cf40b09f9c49919348a968" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33445 } }
I20260806 06:40:14.219112 28661 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a35c98ba02cf40b09f9c49919348a968. Latest consensus state: current_term: 1 leader_uuid: "a35c98ba02cf40b09f9c49919348a968" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35c98ba02cf40b09f9c49919348a968" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33445 } } }
I20260806 06:40:14.219240 28661 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:14.219408 28660 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a35c98ba02cf40b09f9c49919348a968" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35c98ba02cf40b09f9c49919348a968" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 33445 } } }
I20260806 06:40:14.219566 28660 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:14.219880 28682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:14.220511 28682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:14.220965 28657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:14.219482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38709 (local address 127.29.142.190:33445)
0806 06:40:14.219531 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:14.219534 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:14.219706 (+ 172us) server_negotiation.cc:407] Connection header received
0806 06:40:14.219779 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:14.219780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:14.219787 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:14.219855 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:14.220724 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:14.220726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:14.220726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:14.220730 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:14.220748 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:14.220753 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:14.220818 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:14.220890 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 06:40:14.220905 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:40:14.222419 28682 catalog_manager.cc:1347] Generated new cluster ID: e4c2c81251864762bdd45bbbc2a7d733
I20260806 06:40:14.222474 28682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:40:14.232420 28682 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:14.232631 28682 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:14.237026 28682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a35c98ba02cf40b09f9c49919348a968: Generated new TSK 0
I20260806 06:40:14.237241 28682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:14.241240 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 28566
2026-08-06T06:40:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:14 dist-test-slave-3kk6 krb5kdc[28689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:14 dist-test-slave-3kk6 krb5kdc[28689](info): set up 2 sockets
Aug 06 06:40:14 dist-test-slave-3kk6 krb5kdc[28689](info): commencing operation
krb5kdc: starting...
W20260806 06:40:15.459153 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:15 dist-test-slave-3kk6 krb5kdc[28689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:40:15.587827 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:40253
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:40253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:15.673839 28705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:15.674115 28705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:15.674158 28705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:15.675369 28705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:15.675446 28705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:15.675470 28705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:15.675493 28705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:15.675522 28705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:15.677052 28705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:40253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:40253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:15.677356 28705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:15.677604 28705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:15.682849 28714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:15.682859 28716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:15.682859 28712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:15.683111 28705 server_base.cc:1034] running on GCE node
I20260806 06:40:15.683403 28705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:15.683693 28705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:15.684873 28705 hybrid_clock.cc:648] HybridClock initialized: now 1785998415684859 us; error 31 us; skew 500 ppm
Aug 06 06:40:15 dist-test-slave-3kk6 krb5kdc[28689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998415, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:15.687515 28705 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:40:15.687937 28705 webserver.cc:460] Webserver started at http://127.29.142.190:37499/ using document root <none> and password file <none>
I20260806 06:40:15.688144 28705 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:15.688186 28705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:15.688290 28705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:15.689333 28705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "a06bd0d3d3ec4d51badf4cd682f13f19"
format_stamp: "Formatted at 2026-08-06 06:40:15 on dist-test-slave-3kk6"
server_key: "a053bbea514efdafb3fe1f207757e7d0"
server_key_iv: "9223204cf8c90c44fe98bd5dfca26bf6"
server_key_version: "encryptionkey@0"
I20260806 06:40:15.689659 28705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "a06bd0d3d3ec4d51badf4cd682f13f19"
format_stamp: "Formatted at 2026-08-06 06:40:15 on dist-test-slave-3kk6"
server_key: "a053bbea514efdafb3fe1f207757e7d0"
server_key_iv: "9223204cf8c90c44fe98bd5dfca26bf6"
server_key_version: "encryptionkey@0"
I20260806 06:40:15.691110 28705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:40:15.691871 28728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:15.692075 28705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:40:15.692174 28705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "a06bd0d3d3ec4d51badf4cd682f13f19"
format_stamp: "Formatted at 2026-08-06 06:40:15 on dist-test-slave-3kk6"
server_key: "a053bbea514efdafb3fe1f207757e7d0"
server_key_iv: "9223204cf8c90c44fe98bd5dfca26bf6"
server_key_version: "encryptionkey@0"
I20260806 06:40:15.692256 28705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:15.703562 28705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:15.706107 28705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:15.706286 28705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:15.710572 28705 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:40253
I20260806 06:40:15.710609 28798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:40253 every 8 connection(s)
I20260806 06:40:15.711052 28705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:15.711935 28800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:15.713889 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 28705
I20260806 06:40:15.714015 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:15.714190 30266 external_mini_cluster.cc:1428] Setting key 8a7991c07b64d78599d4350a5d7dcdfa
I20260806 06:40:15.714807 28800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19: Bootstrap starting.
I20260806 06:40:15.715567 28800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:15.715922 28800 log.cc:824] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:15.716706 28800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19: No bootstrap required, opened a new log
I20260806 06:40:15.718649 28800 raft_consensus.cc:348] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a06bd0d3d3ec4d51badf4cd682f13f19" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40253 } }
I20260806 06:40:15.718839 28800 raft_consensus.cc:374] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:15.718864 28800 raft_consensus.cc:729] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06bd0d3d3ec4d51badf4cd682f13f19, State: Initialized, Role: FOLLOWER
I20260806 06:40:15.719020 28800 consensus_queue.cc:260] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a06bd0d3d3ec4d51badf4cd682f13f19" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40253 } }
I20260806 06:40:15.719085 28800 raft_consensus.cc:388] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:15.719107 28800 raft_consensus.cc:482] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:15.719177 28800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:15.719858 28800 raft_consensus.cc:504] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a06bd0d3d3ec4d51badf4cd682f13f19" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40253 } }
I20260806 06:40:15.719990 28800 leader_election.cc:302] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a06bd0d3d3ec4d51badf4cd682f13f19; no voters:
I20260806 06:40:15.720194 28800 leader_election.cc:288] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:15.720232 28805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:15.720351 28805 raft_consensus.cc:686] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [term 1 LEADER]: Becoming Leader. State: Replica: a06bd0d3d3ec4d51badf4cd682f13f19, State: Running, Role: LEADER
I20260806 06:40:15.720495 28800 sys_catalog.cc:564] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:15.720495 28805 consensus_queue.cc:237] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a06bd0d3d3ec4d51badf4cd682f13f19" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40253 } }
I20260806 06:40:15.720887 28805 sys_catalog.cc:455] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a06bd0d3d3ec4d51badf4cd682f13f19. Latest consensus state: current_term: 1 leader_uuid: "a06bd0d3d3ec4d51badf4cd682f13f19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a06bd0d3d3ec4d51badf4cd682f13f19" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40253 } } }
I20260806 06:40:15.720974 28805 sys_catalog.cc:458] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:15.720911 28807 sys_catalog.cc:455] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a06bd0d3d3ec4d51badf4cd682f13f19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a06bd0d3d3ec4d51badf4cd682f13f19" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40253 } } }
I20260806 06:40:15.721148 28807 sys_catalog.cc:458] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:15.721215 28813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 06:40:15 dist-test-slave-3kk6 krb5kdc[28689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:40:15.721781 28813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:15.724089 28813 catalog_manager.cc:1347] Generated new cluster ID: 568fc585f6024c06a9b0c4236183794c
I20260806 06:40:15.724146 28813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:40:15.726329 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:15.714809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55119 (local address 127.29.142.190:40253)
0806 06:40:15.715065 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:15.715074 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:15.715106 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:40:15.715768 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:15.715779 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:15.715849 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:15.716013 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:15.716894 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:15.717790 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:15.718590 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:15.719245 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:15.719372 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:15.722300 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:15.722305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:15.722305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:15.722318 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:15.724535 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:15.724980 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:15.724984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:15.724984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:15.725097 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:15.725319 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:15.725320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:15.725320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:15.725658 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:15.725817 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:15.725947 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 06:40:15.726100 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":105,"threads_started":1}
I20260806 06:40:15.727438 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:15.738051 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:15.730454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38247 (local address 127.29.142.190:40253)
0806 06:40:15.730496 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:15.730498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:15.730673 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:40:15.730746 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:15.730747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:15.730758 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:15.730813 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:15.731476 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:15.732233 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:15.733274 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:15.733579 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:15.733640 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:15.735257 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:15.735258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:15.735259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:15.735266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:15.736881 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:15.737279 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:15.737280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:15.737280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:15.737336 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:15.737535 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:15.737535 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:15.737536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:15.737655 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:15.737747 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:15.737876 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 06:40:15.737955 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:40:15.743815 28813 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:15.744563 28813 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:15.749675 28813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a06bd0d3d3ec4d51badf4cd682f13f19: Generated new TSK 0
I20260806 06:40:15.749889 28813 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:15.759166 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 28705
2026-08-06T06:40:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:15 dist-test-slave-3kk6 krb5kdc[28842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:15 dist-test-slave-3kk6 krb5kdc[28842](info): set up 2 sockets
Aug 06 06:40:15 dist-test-slave-3kk6 krb5kdc[28842](info): commencing operation
krb5kdc: starting...
W20260806 06:40:16.978420 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:17 dist-test-slave-3kk6 krb5kdc[28842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:40:17.105659 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:40545
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:41321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:40545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:17.190235 28856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:17.190531 28856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:17.190575 28856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:17.191841 28856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:17.191931 28856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:17.191958 28856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:17.191987 28856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:17.192010 28856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:17.193604 28856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:41321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:40545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:40545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:17.193939 28856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:17.194201 28856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:17.199465 28865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:17.199515 28864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:17.199525 28867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:17.199661 28856 server_base.cc:1034] running on GCE node
I20260806 06:40:17.200048 28856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:17.200435 28856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:17.201790 28856 hybrid_clock.cc:648] HybridClock initialized: now 1785998417201764 us; error 42 us; skew 500 ppm
Aug 06 06:40:17 dist-test-slave-3kk6 krb5kdc[28842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998417, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:17.209136 28856 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:40:17.209743 28856 webserver.cc:460] Webserver started at http://127.29.142.190:33909/ using document root <none> and password file <none>
I20260806 06:40:17.210008 28856 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:17.210085 28856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:17.210251 28856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:17.211531 28856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "6ac992d87039483dbc01c4706eaffb19"
format_stamp: "Formatted at 2026-08-06 06:40:17 on dist-test-slave-3kk6"
server_key: "d9d56efc869ca0dc23b1d06754300de0"
server_key_iv: "e02fbd4ad1047f0d6b30f73116ac51c4"
server_key_version: "encryptionkey@0"
I20260806 06:40:17.211975 28856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "6ac992d87039483dbc01c4706eaffb19"
format_stamp: "Formatted at 2026-08-06 06:40:17 on dist-test-slave-3kk6"
server_key: "d9d56efc869ca0dc23b1d06754300de0"
server_key_iv: "e02fbd4ad1047f0d6b30f73116ac51c4"
server_key_version: "encryptionkey@0"
I20260806 06:40:17.213506 28856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:40:17.214326 28875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:17.214551 28856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:40:17.214708 28856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "6ac992d87039483dbc01c4706eaffb19"
format_stamp: "Formatted at 2026-08-06 06:40:17 on dist-test-slave-3kk6"
server_key: "d9d56efc869ca0dc23b1d06754300de0"
server_key_iv: "e02fbd4ad1047f0d6b30f73116ac51c4"
server_key_version: "encryptionkey@0"
I20260806 06:40:17.214826 28856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:17.234705 28856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:17.237479 28856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:17.237680 28856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:17.242285 28856 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:40545
I20260806 06:40:17.242339 28950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:40545 every 8 connection(s)
I20260806 06:40:17.242760 28856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:17.243680 28952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:17.246088 28952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19: Bootstrap starting.
I20260806 06:40:17.246834 28952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:17.247153 28952 log.cc:824] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:17.247932 28952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19: No bootstrap required, opened a new log
I20260806 06:40:17.249562 28952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac992d87039483dbc01c4706eaffb19" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40545 } }
I20260806 06:40:17.249711 28952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:17.249735 28952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ac992d87039483dbc01c4706eaffb19, State: Initialized, Role: FOLLOWER
I20260806 06:40:17.249936 28952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac992d87039483dbc01c4706eaffb19" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40545 } }
I20260806 06:40:17.250005 28952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:17.250046 28952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:17.250100 28952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:17.251123 28952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac992d87039483dbc01c4706eaffb19" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40545 } }
I20260806 06:40:17.251242 28952 leader_election.cc:302] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ac992d87039483dbc01c4706eaffb19; no voters:
I20260806 06:40:17.251484 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 28856
I20260806 06:40:17.251505 28952 leader_election.cc:288] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:17.251534 28955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:17.251621 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:17.251675 28955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [term 1 LEADER]: Becoming Leader. State: Replica: 6ac992d87039483dbc01c4706eaffb19, State: Running, Role: LEADER
I20260806 06:40:17.251837 28952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:17.251874 30266 external_mini_cluster.cc:1428] Setting key f3ff44d6acb68af6099bfa4d7e1a27ca
I20260806 06:40:17.251844 28955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac992d87039483dbc01c4706eaffb19" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40545 } }
I20260806 06:40:17.252373 28957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ac992d87039483dbc01c4706eaffb19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac992d87039483dbc01c4706eaffb19" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40545 } } }
I20260806 06:40:17.252422 28956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ac992d87039483dbc01c4706eaffb19. Latest consensus state: current_term: 1 leader_uuid: "6ac992d87039483dbc01c4706eaffb19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac992d87039483dbc01c4706eaffb19" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 40545 } } }
I20260806 06:40:17.252530 28957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:17.252548 28956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19 [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:17.253191 28963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:17.253831 28963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:17.255960 28963 catalog_manager.cc:1347] Generated new cluster ID: 085a5f415e5f4c5686aaed0cb4b45ef6
I20260806 06:40:17.256023 28963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:40:17.258352 28962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:17.252662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55423 (local address 127.29.142.190:40545)
0806 06:40:17.252828 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:17.252839 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:17.253111 (+ 272us) server_negotiation.cc:407] Connection header received
0806 06:40:17.253266 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:17.253274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:17.253349 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:17.253496 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:17.253862 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:17.255226 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:17.256345 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:17.256768 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:17.256894 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:17.257660 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:17.257664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:17.257665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:17.257853 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:17.257885 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:17.257896 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:17.257975 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:17.258057 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 06:40:17.258128 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260806 06:40:17.259370 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:17.263707 28962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:17.260816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35715 (local address 127.29.142.190:40545)
0806 06:40:17.260856 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:17.260858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:17.260949 (+ 91us) server_negotiation.cc:407] Connection header received
0806 06:40:17.261008 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:17.261009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:17.261031 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:17.261087 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:17.261208 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:17.261685 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:17.262396 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:17.262606 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:17.262700 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:17.263450 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:17.263452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:17.263452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:17.263457 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:17.263471 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:17.263475 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:17.263523 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:17.263581 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 06:40:17.263605 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:40:17.286051 28963 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:17.286878 28963 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:17.290926 28963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ac992d87039483dbc01c4706eaffb19: Generated new TSK 0
I20260806 06:40:17.291168 28963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:17.295341 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 28856
2026-08-06T06:40:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1543 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:17 dist-test-slave-3kk6 krb5kdc[28995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:17 dist-test-slave-3kk6 krb5kdc[28995](info): set up 2 sockets
Aug 06 06:40:17 dist-test-slave-3kk6 krb5kdc[28995](info): commencing operation
krb5kdc: starting...
W20260806 06:40:18.526055 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:18 dist-test-slave-3kk6 krb5kdc[28995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:40:18.654250 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:41533
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:39041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:41533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:18.739396 29013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:18.739683 29013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:18.739737 29013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:18.740955 29013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:18.741036 29013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:18.741062 29013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:18.741096 29013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:18.741139 29013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:18.742776 29013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:39041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:41533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:41533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:18.743074 29013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:18.743328 29013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:18.749130 29020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:18.749171 29019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:18.749298 29013 server_base.cc:1034] running on GCE node
W20260806 06:40:18.749188 29024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:18.749635 29013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:18.750052 29013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:18.751242 29013 hybrid_clock.cc:648] HybridClock initialized: now 1785998418751236 us; error 63 us; skew 500 ppm
Aug 06 06:40:18 dist-test-slave-3kk6 krb5kdc[28995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998418, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:18.758339 29013 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:40:18.759008 29013 webserver.cc:460] Webserver started at http://127.29.142.190:35757/ using document root <none> and password file <none>
I20260806 06:40:18.759312 29013 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:18.759390 29013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:18.759555 29013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:18.760715 29013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "f9b50d25f517432fb85582604de136bd"
format_stamp: "Formatted at 2026-08-06 06:40:18 on dist-test-slave-3kk6"
server_key: "c241b0be7baceef3545f5675cb1bd623"
server_key_iv: "dca1e9b84804c27e80889eaa613af04c"
server_key_version: "encryptionkey@0"
I20260806 06:40:18.761112 29013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "f9b50d25f517432fb85582604de136bd"
format_stamp: "Formatted at 2026-08-06 06:40:18 on dist-test-slave-3kk6"
server_key: "c241b0be7baceef3545f5675cb1bd623"
server_key_iv: "dca1e9b84804c27e80889eaa613af04c"
server_key_version: "encryptionkey@0"
I20260806 06:40:18.762588 29013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:40:18.763482 29035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:18.763700 29013 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:40:18.763799 29013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "f9b50d25f517432fb85582604de136bd"
format_stamp: "Formatted at 2026-08-06 06:40:18 on dist-test-slave-3kk6"
server_key: "c241b0be7baceef3545f5675cb1bd623"
server_key_iv: "dca1e9b84804c27e80889eaa613af04c"
server_key_version: "encryptionkey@0"
I20260806 06:40:18.763886 29013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:18.767575 29013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:18.770190 29013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:18.770372 29013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:18.774885 29013 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:41533
I20260806 06:40:18.774917 29106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:41533 every 8 connection(s)
I20260806 06:40:18.775303 29013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:18.776360 29108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:18.778947 29108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd: Bootstrap starting.
I20260806 06:40:18.779613 29108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:18.779947 29108 log.cc:824] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:18.780033 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 29013
I20260806 06:40:18.780224 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:18.780421 30266 external_mini_cluster.cc:1428] Setting key e86b9a945186c4d97e757c5fe131fc09
I20260806 06:40:18.780840 29108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd: No bootstrap required, opened a new log
I20260806 06:40:18.783354 29108 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b50d25f517432fb85582604de136bd" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41533 } }
I20260806 06:40:18.783591 29108 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:40:18.783658 29108 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9b50d25f517432fb85582604de136bd, State: Initialized, Role: FOLLOWER
I20260806 06:40:18.783802 29108 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b50d25f517432fb85582604de136bd" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41533 } }
I20260806 06:40:18.783900 29108 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:40:18.783946 29108 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:40:18.783998 29108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:18.784000 29116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:18.780996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39767 (local address 127.29.142.190:41533)
0806 06:40:18.781208 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:18.781217 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:18.781245 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:40:18.782180 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:18.782189 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:18.782219 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:18.782383 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:18.783185 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:18.783188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:18.783188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:18.783193 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:18.783240 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:18.783250 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:18.783319 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:18.783418 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 06:40:18.783476 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":111,"threads_started":1}
I20260806 06:40:18.784924 29108 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b50d25f517432fb85582604de136bd" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41533 } }
I20260806 06:40:18.785039 29108 leader_election.cc:302] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9b50d25f517432fb85582604de136bd; no voters:
I20260806 06:40:18.785111 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:18.785254 29108 leader_election.cc:288] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:40:18.785310 29117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:18.785482 29117 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [term 1 LEADER]: Becoming Leader. State: Replica: f9b50d25f517432fb85582604de136bd, State: Running, Role: LEADER
I20260806 06:40:18.785512 29108 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:18.785575 29117 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b50d25f517432fb85582604de136bd" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41533 } }
I20260806 06:40:18.785887 29117 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9b50d25f517432fb85582604de136bd. Latest consensus state: current_term: 1 leader_uuid: "f9b50d25f517432fb85582604de136bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b50d25f517432fb85582604de136bd" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41533 } } }
I20260806 06:40:18.785894 29118 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9b50d25f517432fb85582604de136bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9b50d25f517432fb85582604de136bd" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 41533 } } }
I20260806 06:40:18.785979 29117 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:18.785980 29118 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:18.787104 29125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:18.788172 29125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:18.788419 29116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:18.787042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45971 (local address 127.29.142.190:41533)
0806 06:40:18.787080 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:18.787083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:18.787141 (+ 58us) server_negotiation.cc:407] Connection header received
0806 06:40:18.787257 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:18.787258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:18.787265 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:18.787309 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:18.788101 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:18.788103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:18.788103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:40:18.788108 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:18.788126 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:40:18.788131 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:18.788205 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:18.788335 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 06:40:18.788352 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:40:18.789978 29125 catalog_manager.cc:1347] Generated new cluster ID: 709038b2a94645b7989eef3a3136bbe5
I20260806 06:40:18.790035 29125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:40:18.809729 29125 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:18.809978 29125 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:18.814970 29125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9b50d25f517432fb85582604de136bd: Generated new TSK 0
I20260806 06:40:18.815220 29125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:18.818256 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 29013
2026-08-06T06:40:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1517 ms)
[----------] 18 tests from AuthTokenIssuingTest (27735 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:18 dist-test-slave-3kk6 krb5kdc[29157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:18 dist-test-slave-3kk6 krb5kdc[29157](info): set up 2 sockets
Aug 06 06:40:18 dist-test-slave-3kk6 krb5kdc[29157](info): commencing operation
krb5kdc: starting...
W20260806 06:40:20.056203 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:40:20.190003 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:37889
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:38621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:37889,127.29.142.189:34039,127.29.142.188:42847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:20.277613 29174 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:40:20.277875 29174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:20.277894 29174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:20.277916 29174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:20.279170 29174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:20.279223 29174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:20.279238 29174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:20.279253 29174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:20.279265 29174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:20.280870 29174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:38621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:37889,127.29.142.189:34039,127.29.142.188:42847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:37889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:20.281162 29174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:20.281442 29174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:20.286795 29182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:20.286782 29180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:20.286940 29174 server_base.cc:1034] running on GCE node
W20260806 06:40:20.287082 29184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:20.287377 29174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:20.287707 29174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:20.288888 29174 hybrid_clock.cc:648] HybridClock initialized: now 1785998420288864 us; error 48 us; skew 500 ppm
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:20.291716 29174 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:40:20.292258 29174 webserver.cc:460] Webserver started at http://127.29.142.190:45167/ using document root <none> and password file <none>
I20260806 06:40:20.292515 29174 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:20.292588 29174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:20.292752 29174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:20.293926 29174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "ff37261ca0394b2da8f2840becb2b72b"
format_stamp: "Formatted at 2026-08-06 06:40:20 on dist-test-slave-3kk6"
server_key: "b65b1967cc33b55104a27b94f5e49fd9"
server_key_iv: "5a042411de75b0e4794f54c4a768ff23"
server_key_version: "encryptionkey@0"
I20260806 06:40:20.294335 29174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "ff37261ca0394b2da8f2840becb2b72b"
format_stamp: "Formatted at 2026-08-06 06:40:20 on dist-test-slave-3kk6"
server_key: "b65b1967cc33b55104a27b94f5e49fd9"
server_key_iv: "5a042411de75b0e4794f54c4a768ff23"
server_key_version: "encryptionkey@0"
I20260806 06:40:20.295892 29174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:40:20.296703 29195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:20.296890 29174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:40:20.297004 29174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "ff37261ca0394b2da8f2840becb2b72b"
format_stamp: "Formatted at 2026-08-06 06:40:20 on dist-test-slave-3kk6"
server_key: "b65b1967cc33b55104a27b94f5e49fd9"
server_key_iv: "5a042411de75b0e4794f54c4a768ff23"
server_key_version: "encryptionkey@0"
I20260806 06:40:20.297084 29174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:20.310531 29174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:20.312702 29174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:20.312904 29174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:20.317514 29174 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:37889
I20260806 06:40:20.317554 29271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:37889 every 8 connection(s)
I20260806 06:40:20.317936 29174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:20.318933 29272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:20.320508 29272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:20.326043 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 29174
I20260806 06:40:20.326156 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:20.326392 30266 external_mini_cluster.cc:1428] Setting key 9c71334de6199f7b2e8851bedfceb5f3
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:40:20.331321 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.321209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37889 (local address 127.0.0.1:52920)
0806 06:40:20.321393 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.321436 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:20.321943 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:20.322245 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:20.322248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:20.322502 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:20.322692 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.322696 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.323746 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.323747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.324444 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.324446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.324952 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.324954 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.325019 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.325555 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:20.325564 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:20.327383 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:20.329533 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.329547 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.329550 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.329785 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.330077 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.330078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.330078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.330165 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.330716 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:20.330722 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:20.330791 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 06:40:20.330872 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:40:20.331017 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":100,"threads_started":1}
I20260806 06:40:20.331326 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.321266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52920 (local address 127.29.142.190:37889)
0806 06:40:20.321442 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.321446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.321543 (+ 97us) server_negotiation.cc:407] Connection header received
0806 06:40:20.322082 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.322084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.322101 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.322194 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.322813 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.323644 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.324525 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.324839 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.324925 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.327506 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.327515 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.327516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.327523 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.329423 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.329877 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.329880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.329881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.330000 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.330244 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.330245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.330246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.330583 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.330739 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.330939 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.331114 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260806 06:40:20.332242 29272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:20.332660 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.332502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34039 (local address 127.0.0.1:52964)
0806 06:40:20.332566 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.332626 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 06:40:20.332855 29197 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34039: connect: Connection refused (error 111)
I20260806 06:40:20.333352 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.333230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34039 (local address 127.0.0.1:52968)
0806 06:40:20.333284 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.333320 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:40:20.333509 29272 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.189:34039: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34039: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.29.142.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.189.keytab.
Entry for principal kudu/127.29.142.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.189.keytab.
I20260806 06:40:20.353655 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.189:34039
--webserver_interface=127.29.142.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.189.keytab
--principal=kudu/127.29.142.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:38621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:37889,127.29.142.189:34039,127.29.142.188:42847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:40:20.382745 29272 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } attempt: 1
I20260806 06:40:20.383350 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.383196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34039 (local address 127.0.0.1:52972)
0806 06:40:20.383265 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.383305 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 06:40:20.383714 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.383606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34039 (local address 127.0.0.1:52976)
0806 06:40:20.383659 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.383688 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:40:20.383823 29272 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.189:34039: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34039: connect: Connection refused (error 111)
W20260806 06:40:20.440361 29282 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:40:20.440620 29282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:20.440663 29282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:20.440713 29282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:20.441859 29282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:20.441927 29282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:20.441956 29282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:20.441988 29282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:20.442016 29282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:20.443583 29282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:38621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:37889,127.29.142.189:34039,127.29.142.188:42847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.189.keytab
--principal=kudu/127.29.142.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.189:34039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.142.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:20.443864 29282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:20.444110 29282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:20.449059 29292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:20.449082 29290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:20.449054 29288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:20.449342 29282 server_base.cc:1034] running on GCE node
I20260806 06:40:20.449590 29282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:20.449900 29282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:20.451068 29282 hybrid_clock.cc:648] HybridClock initialized: now 1785998420451065 us; error 40 us; skew 500 ppm
I20260806 06:40:20.451982 29272 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } attempt: 2
I20260806 06:40:20.452397 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.452259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34039 (local address 127.0.0.1:52992)
0806 06:40:20.452328 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.452365 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:40:20.452749 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.452688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34039 (local address 127.0.0.1:53004)
0806 06:40:20.452713 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.452734 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 06:40:20.452859 29272 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.189:34039: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34039: connect: Connection refused (error 111)
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:20.453974 29282 init.cc:377] Logged in from keytab as kudu/127.29.142.189@KRBTEST.COM (short username kudu)
I20260806 06:40:20.454383 29282 webserver.cc:460] Webserver started at http://127.29.142.189:42021/ using document root <none> and password file <none>
I20260806 06:40:20.454591 29282 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:20.454633 29282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:20.454809 29282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:20.455868 29282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/data/instance:
uuid: "24fb5cfb03274726bdf7aab7e95b6814"
format_stamp: "Formatted at 2026-08-06 06:40:20 on dist-test-slave-3kk6"
server_key: "65dc99594ce67e601eab51eadb47f577"
server_key_iv: "c470577c2749b2e071b5327b1c7d5525"
server_key_version: "encryptionkey@0"
I20260806 06:40:20.456192 29282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/wal/instance:
uuid: "24fb5cfb03274726bdf7aab7e95b6814"
format_stamp: "Formatted at 2026-08-06 06:40:20 on dist-test-slave-3kk6"
server_key: "65dc99594ce67e601eab51eadb47f577"
server_key_iv: "c470577c2749b2e071b5327b1c7d5525"
server_key_version: "encryptionkey@0"
I20260806 06:40:20.457545 29282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:40:20.458273 29304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:20.458474 29282 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:40:20.458587 29282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/wal
uuid: "24fb5cfb03274726bdf7aab7e95b6814"
format_stamp: "Formatted at 2026-08-06 06:40:20 on dist-test-slave-3kk6"
server_key: "65dc99594ce67e601eab51eadb47f577"
server_key_iv: "c470577c2749b2e071b5327b1c7d5525"
server_key_version: "encryptionkey@0"
I20260806 06:40:20.458693 29282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:20.485131 29282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:20.487424 29282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:20.487607 29282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:20.492054 29282 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.189:34039
I20260806 06:40:20.492095 29372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.189:34039 every 8 connection(s)
I20260806 06:40:20.492532 29282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
I20260806 06:40:20.493446 29374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:20.495190 29374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:20.499614 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 29282
I20260806 06:40:20.499737 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-1/wal/instance
I20260806 06:40:20.500041 30266 external_mini_cluster.cc:1428] Setting key 4ff6b37366cc544a34817bc0f16ddf5d
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.189@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:40:20.505417 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.495948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52932 (local address 127.29.142.190:37889)
0806 06:40:20.496002 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.496004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.496213 (+ 209us) server_negotiation.cc:407] Connection header received
0806 06:40:20.496666 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.496667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.496677 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.496739 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.497320 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.497814 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.498827 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.499026 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.499085 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.501895 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.501897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.501898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.501911 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.503742 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.504161 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.504163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.504163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.504240 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.504573 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.504574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.504575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.504756 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.504840 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.505153 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.505257 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:40:20.505432 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.495900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37889 (local address 127.0.0.1:52932)
0806 06:40:20.496073 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.496116 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:20.496586 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:20.496791 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:20.496794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:20.497042 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:20.497243 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.497249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.497898 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.497899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.498751 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.498754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.499103 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.499104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.499214 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.499875 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:20.499889 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:20.501761 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:20.503838 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.503845 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.503847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.504080 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.504347 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.504355 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.504355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.504486 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.504860 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:20.504867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:20.504996 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 06:40:20.505110 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:40:20.505258 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260806 06:40:20.506095 29374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.29.142.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.188@KRBTEST.COM" created.
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.189@KRBTEST.COM for kudu/127.29.142.189@KRBTEST.COM
I20260806 06:40:20.515676 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.506388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34039 (local address 127.0.0.1:53006)
0806 06:40:20.506617 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.506628 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:20.506716 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:20.506856 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:20.506856 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:20.507004 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:20.507055 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.507056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.507964 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.507965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.508462 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.508463 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.508960 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.508961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.509024 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.509489 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:20.509494 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:20.512533 (+ 3039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:20.514520 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.514521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.514522 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.514809 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.515042 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.515043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.515043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.515100 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.515447 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:20.515447 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:20.515483 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 06:40:20.515529 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:40:20.515604 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:40:20.515928 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.506472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53006 (local address 127.29.142.189:34039)
0806 06:40:20.506634 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.506638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.506725 (+ 87us) server_negotiation.cc:407] Connection header received
0806 06:40:20.506774 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.506777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.506791 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.506857 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.507135 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.507854 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.508561 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.508882 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.508965 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.512632 (+ 3667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.512642 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.512642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.512651 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.514420 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.514903 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.514907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.514907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.514972 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.515171 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.515172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.515173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.515379 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.515468 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.515592 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.515683 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":105,"threads_started":1}
I20260806 06:40:20.516557 29374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:20.516959 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.516837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41654)
0806 06:40:20.516887 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.516941 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 06:40:20.517143 29308 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
I20260806 06:40:20.517609 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.517526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41666)
0806 06:40:20.517569 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.517594 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:40:20.517747 29374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:42847: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.188.keytab.
Entry for principal kudu/127.29.142.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.188.keytab.
I20260806 06:40:20.529196 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.188:42847
--webserver_interface=127.29.142.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.188.keytab
--principal=kudu/127.29.142.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:38621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:37889,127.29.142.189:34039,127.29.142.188:42847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:40:20.544031 29272 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } attempt: 3
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for kudu/127.29.142.189@KRBTEST.COM
I20260806 06:40:20.552541 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.544505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34039 (local address 127.0.0.1:53022)
0806 06:40:20.544571 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.544592 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:20.544673 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:20.544858 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:20.544859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:20.545041 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:20.545110 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.545112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.546231 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.546232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.546908 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.546911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.547481 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.547482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.547559 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.548106 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:20.548112 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:20.549612 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:20.551545 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.551546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.551546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.551738 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.551955 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.551956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.551956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.552014 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.552283 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:20.552284 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:20.552327 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 06:40:20.552381 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:40:20.552466 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:40:20.552604 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.544587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53022 (local address 127.29.142.189:34039)
0806 06:40:20.544669 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.544671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.544684 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:40:20.544741 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.544742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.544753 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.544851 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.545255 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.546080 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.547021 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.547375 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.547458 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.549720 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.549721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.549722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.549731 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.551452 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.551820 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.551821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.551821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.551886 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.552076 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.552076 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.552077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.552211 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.552277 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.552426 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.552510 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 06:40:20.553072 29272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:20.553472 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.553352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41670)
0806 06:40:20.553410 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.553449 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:40:20.553817 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.553735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41686)
0806 06:40:20.553757 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.553802 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260806 06:40:20.554033 29272 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:42847: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
I20260806 06:40:20.567004 29374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } attempt: 1
I20260806 06:40:20.567524 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.567382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41688)
0806 06:40:20.567447 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.567494 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 06:40:20.567906 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.567798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41690)
0806 06:40:20.567846 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.567885 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 06:40:20.567992 29374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:42847: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
I20260806 06:40:20.585218 29272 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } attempt: 1
I20260806 06:40:20.585821 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.585670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41696)
0806 06:40:20.585740 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.585790 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 06:40:20.586223 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.586134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41702)
0806 06:40:20.586181 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.586207 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:40:20.586349 29272 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:42847: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
W20260806 06:40:20.619503 29384 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:40:20.619791 29384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:20.619827 29384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:20.619869 29384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:20.621042 29384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:20.621116 29384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:20.621142 29384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:20.621174 29384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:20.621223 29384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:20.622853 29384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:38621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:37889,127.29.142.189:34039,127.29.142.188:42847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.188.keytab
--principal=kudu/127.29.142.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.188:42847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.142.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:20.623152 29384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:20.623412 29384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:20.628746 29391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:20.628831 29393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:20.628754 29390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:20.629092 29384 server_base.cc:1034] running on GCE node
I20260806 06:40:20.629423 29384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:20.629820 29384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:20.630998 29384 hybrid_clock.cc:648] HybridClock initialized: now 1785998420630979 us; error 31 us; skew 500 ppm
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:20.633692 29384 init.cc:377] Logged in from keytab as kudu/127.29.142.188@KRBTEST.COM (short username kudu)
I20260806 06:40:20.634199 29384 webserver.cc:460] Webserver started at http://127.29.142.188:33833/ using document root <none> and password file <none>
I20260806 06:40:20.634469 29384 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:20.634541 29384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:20.634752 29384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:20.635903 29384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/data/instance:
uuid: "e66c2241a2e94da8927d933f8ef52ddf"
format_stamp: "Formatted at 2026-08-06 06:40:20 on dist-test-slave-3kk6"
server_key: "c34a534e3ad1de0c92148904699c1e94"
server_key_iv: "f49b72c0cb2aa8b02fdb7f8eb8e10dff"
server_key_version: "encryptionkey@0"
I20260806 06:40:20.636194 29374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } attempt: 2
I20260806 06:40:20.636355 29384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/wal/instance:
uuid: "e66c2241a2e94da8927d933f8ef52ddf"
format_stamp: "Formatted at 2026-08-06 06:40:20 on dist-test-slave-3kk6"
server_key: "c34a534e3ad1de0c92148904699c1e94"
server_key_iv: "f49b72c0cb2aa8b02fdb7f8eb8e10dff"
server_key_version: "encryptionkey@0"
I20260806 06:40:20.636641 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.636525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41714)
0806 06:40:20.636575 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.636614 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 06:40:20.636993 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.636908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41722)
0806 06:40:20.636952 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.636977 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:40:20.637115 29374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:42847: Network error: Client connection negotiation failed: client connection to 127.29.142.188:42847: connect: Connection refused (error 111)
I20260806 06:40:20.638065 29384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:40:20.638913 29408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:20.639097 29384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:40:20.639204 29384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/wal
uuid: "e66c2241a2e94da8927d933f8ef52ddf"
format_stamp: "Formatted at 2026-08-06 06:40:20 on dist-test-slave-3kk6"
server_key: "c34a534e3ad1de0c92148904699c1e94"
server_key_iv: "f49b72c0cb2aa8b02fdb7f8eb8e10dff"
server_key_version: "encryptionkey@0"
I20260806 06:40:20.639293 29384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:20.653182 29384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:20.655494 29384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:20.655678 29384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:20.659866 29384 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.188:42847
I20260806 06:40:20.659905 29483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.188:42847 every 8 connection(s)
I20260806 06:40:20.660288 29384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
I20260806 06:40:20.661275 29484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:20.661480 29272 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } attempt: 2
I20260806 06:40:20.663539 29484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:20.665228 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 29384
I20260806 06:40:20.665354 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998137760399-30266-0/minicluster-data/master-2/wal/instance
I20260806 06:40:20.665555 30266 external_mini_cluster.cc:1428] Setting key e960796410fbf426b83ea32e43b634be
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for kudu/127.29.142.188@KRBTEST.COM
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.188@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.188@KRBTEST.COM
I20260806 06:40:20.672849 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.661855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41730)
0806 06:40:20.661922 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.661935 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:20.661991 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:20.663076 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:20.663078 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:20.663280 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:20.663326 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.663328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.664734 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.664736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.665433 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.665435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.666226 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.666227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.666293 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.666829 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:20.666835 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:20.668324 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:20.671069 (+ 2745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.671070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.671071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.671317 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.671577 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.671578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.671579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.671658 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.672467 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:20.672468 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:20.672516 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 06:40:20.672610 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 06:40:20.672722 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:40:20.673031 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.664261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.29.142.190:37889)
0806 06:40:20.664321 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.664324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.664475 (+ 151us) server_negotiation.cc:407] Connection header received
0806 06:40:20.664556 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.664557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.664567 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.664627 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.665129 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.665766 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.667056 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.667315 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.667377 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.669805 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.669806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.669807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.669812 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.671634 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.672039 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.672041 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.672041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.672101 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.672310 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.672311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.672312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.672533 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.672616 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.672831 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.672915 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:40:20.673328 29487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.664198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:37889 (local address 127.0.0.1:52946)
0806 06:40:20.664375 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.664408 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:20.664495 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:20.664647 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:20.664649 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:20.664937 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:20.665005 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.665006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.665864 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.665865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.666968 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.666971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.667436 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.667437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.667516 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.668045 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:20.668054 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:20.669715 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:20.671721 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.671728 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.671731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.671942 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.672177 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.672178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.672178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.672235 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.672615 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:20.672621 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:20.672686 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 06:40:20.672793 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 06:40:20.673066 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260806 06:40:20.673903 29486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.662017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41730 (local address 127.29.142.188:42847)
0806 06:40:20.662231 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.662240 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.662313 (+ 73us) server_negotiation.cc:407] Connection header received
0806 06:40:20.662877 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.662885 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.662936 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.663091 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.663451 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.664629 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.665517 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.666138 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.666258 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.668419 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.668427 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.668427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.668434 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.670965 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.671405 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.671408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.671409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.671503 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.672045 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.672046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.672046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.672389 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.672484 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.673489 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.673580 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
I20260806 06:40:20.673997 29484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:20.676285 29272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b: Bootstrap starting.
I20260806 06:40:20.676527 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.665932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41740 (local address 127.29.142.188:42847)
0806 06:40:20.666081 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.666083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.666231 (+ 148us) server_negotiation.cc:407] Connection header received
0806 06:40:20.666356 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.666357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.666366 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.666430 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.667233 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.667869 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.668993 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.669322 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.669397 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.672696 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.672697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.672698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.672704 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.675214 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.675643 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.675645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.675645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.675707 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.675934 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.675935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.675935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.676050 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.676199 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.676287 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.676374 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260806 06:40:20.677138 29272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:20.677521 29272 log.cc:824] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:20.678514 29272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b: No bootstrap required, opened a new log
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.188@KRBTEST.COM for kudu/127.29.142.189@KRBTEST.COM
I20260806 06:40:20.681048 29272 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } }
I20260806 06:40:20.681228 29272 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff37261ca0394b2da8f2840becb2b72b, State: Initialized, Role: FOLLOWER
I20260806 06:40:20.681330 29272 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } }
I20260806 06:40:20.682035 29272 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:20.682192 29487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.674303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34039 (local address 127.0.0.1:53028)
0806 06:40:20.674417 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.674427 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:20.674478 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:20.674612 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:20.674613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:20.674812 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:20.674846 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.674848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.675603 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.675604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.676431 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.676433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.677100 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.677101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.677161 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.677524 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:20.677529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:20.679187 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:20.680909 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.680910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.680911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.681137 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.681396 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.681397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.681397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.681472 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.681852 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:20.681853 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:20.681906 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 06:40:20.681985 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:40:20.682096 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 06:40:20.682236 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.674369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53028 (local address 127.29.142.189:34039)
0806 06:40:20.674427 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.674429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.674464 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:40:20.674542 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.674544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.674554 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.674601 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.674930 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.675504 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.676533 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.676844 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.676891 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.679264 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.679266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.679266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.679274 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.680805 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.681228 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.681229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.681230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.681307 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.681588 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.681589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.681589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.681758 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.681836 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.682034 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.682114 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:40:20.682396 29492 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } } }
I20260806 06:40:20.682483 29492 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:20.682603 29484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
W20260806 06:40:20.684202 29511 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:40:20.684283 29511 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.188@KRBTEST.COM for kudu/127.29.142.188@KRBTEST.COM
I20260806 06:40:20.688467 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.678358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.29.142.190:37889)
0806 06:40:20.678398 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.678400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.678409 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:40:20.678459 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.678459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.678469 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.678516 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.679176 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.679886 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.680966 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.681171 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.681269 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.684778 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.684780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.684781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.684789 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.687054 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.687488 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.687489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.687489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.687552 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.687754 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.687755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.687755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.687879 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.688037 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.688284 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.688378 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:40:20.687296 29174 master_runner.cc:186] Error getting master registration for 127.29.142.189:34039: OK, instance_id { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" instance_seqno: 1785998420490768 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:40:20.690099 29174 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:40:20.691133 29487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.682897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41754)
0806 06:40:20.682983 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.682995 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:20.683036 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:20.683215 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:20.683216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:20.683405 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:20.683450 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.683452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.684364 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.684366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.685104 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.685108 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.685735 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.685736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.685794 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.686217 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:20.686223 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:20.687702 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:20.689898 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.689899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.689900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.690107 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.690302 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.690303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.690303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.690352 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.690827 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:20.690828 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:20.690866 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 06:40:20.690927 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:40:20.691025 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 06:40:20.691512 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.682986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41754 (local address 127.29.142.188:42847)
0806 06:40:20.683034 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.683036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.683046 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:40:20.683103 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.683104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.683117 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.683208 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.683526 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.684275 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.685272 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.685629 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.685709 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.687804 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.687805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.687805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.687811 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.689805 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.690180 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.690181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.690181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.690228 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.690410 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.690410 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.690410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.690733 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.690804 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.691347 (+ 543us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.691430 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:40:20.693212 29484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf: Bootstrap starting.
I20260806 06:40:20.693984 29484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:20.694348 29484 log.cc:824] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:20.695190 29484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf: No bootstrap required, opened a new log
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.189@KRBTEST.COM
I20260806 06:40:20.697057 29484 raft_consensus.cc:348] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } }
I20260806 06:40:20.697259 29484 raft_consensus.cc:729] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e66c2241a2e94da8927d933f8ef52ddf, State: Initialized, Role: FOLLOWER
I20260806 06:40:20.697357 29484 consensus_queue.cc:260] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } }
I20260806 06:40:20.697758 29484 sys_catalog.cc:564] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:20.698127 29514 sys_catalog.cc:455] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } } }
I20260806 06:40:20.699321 29514 sys_catalog.cc:458] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:40:20.700235 29528 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:40:20.700294 29528 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:40:20.701184 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.690974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53034 (local address 127.29.142.189:34039)
0806 06:40:20.691017 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.691019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.691028 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:40:20.691077 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.691078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.691088 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.691133 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.691831 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.692585 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.693527 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.693847 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.693926 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.697171 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.697172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.697173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.697181 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.699208 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.699726 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.699727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.699728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.699801 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.700092 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.700093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.700094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.700251 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.700450 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.701003 (+ 553us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.701087 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:40:20.701772 29384 master_runner.cc:186] Error getting master registration for 127.29.142.189:34039: OK, instance_id { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" instance_seqno: 1785998420490768 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:40:20.704173 29384 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:40:20.704473 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:20.705570 29230 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ff37261ca0394b2da8f2840becb2b72b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52956
I20260806 06:40:20.705729 29230 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 06:40:20.705786 29230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:20.706846 29230 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } }
I20260806 06:40:20.707104 29230 leader_election.cc:288] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [CANDIDATE]: Term 1 election: Requested vote from peers 24fb5cfb03274726bdf7aab7e95b6814 (127.29.142.189:34039), e66c2241a2e94da8927d933f8ef52ddf (127.29.142.188:42847)
I20260806 06:40:20.707610 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff37261ca0394b2da8f2840becb2b72b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e66c2241a2e94da8927d933f8ef52ddf"
I20260806 06:40:20.707728 29444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:20.708633 29444 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff37261ca0394b2da8f2840becb2b72b in term 1.
I20260806 06:40:20.708861 29198 leader_election.cc:302] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e66c2241a2e94da8927d933f8ef52ddf, ff37261ca0394b2da8f2840becb2b72b; no voters:
I20260806 06:40:20.709038 29492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:20.709232 29492 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [term 1 LEADER]: Becoming Leader. State: Replica: ff37261ca0394b2da8f2840becb2b72b, State: Running, Role: LEADER
I20260806 06:40:20.709342 29492 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } }
I20260806 06:40:20.709791 29536 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff37261ca0394b2da8f2840becb2b72b. Latest consensus state: current_term: 1 leader_uuid: "ff37261ca0394b2da8f2840becb2b72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } } }
I20260806 06:40:20.710033 29536 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:20.710393 29539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:20.710611 29539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:20.712625 29444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [term 1 FOLLOWER]: Refusing update from remote peer ff37261ca0394b2da8f2840becb2b72b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:40:20.713027 29536 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:40:20.714536 29514 sys_catalog.cc:455] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff37261ca0394b2da8f2840becb2b72b. Latest consensus state: current_term: 1 leader_uuid: "ff37261ca0394b2da8f2840becb2b72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } } }
I20260806 06:40:20.714625 29514 sys_catalog.cc:458] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:20.719447 29536 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff37261ca0394b2da8f2840becb2b72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } } }
I20260806 06:40:20.719568 29536 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:20.719949 29514 sys_catalog.cc:455] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff37261ca0394b2da8f2840becb2b72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } } }
I20260806 06:40:20.720062 29514 sys_catalog.cc:458] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:20.723165 29544 mvcc.cc:205] Tried to move back new op lower bound from 7315449531235864576 to 7315449531226185728. Current Snapshot: MvccSnapshot[applied={T|T < 7315449531235864576}]
I20260806 06:40:20.726222 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.715159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41762 (local address 127.29.142.188:42847)
0806 06:40:20.715221 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.715223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.715427 (+ 204us) server_negotiation.cc:407] Connection header received
0806 06:40:20.715653 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.715654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.715689 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.715725 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.717033 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.717804 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.719439 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.719760 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.719804 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.722614 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.722616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.722616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.722624 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.724508 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.725116 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.725117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.725117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.725181 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.725502 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.725503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.725503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.725637 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.725720 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.726040 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.726130 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:40:20.727089 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.714921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52970 (local address 127.29.142.190:37889)
0806 06:40:20.714984 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.714987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.715874 (+ 887us) server_negotiation.cc:407] Connection header received
0806 06:40:20.716091 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.716093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.716106 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.716139 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.717327 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.718053 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.719614 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.719888 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.719937 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.722777 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.722779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.722779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.722787 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.725174 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.725828 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.725841 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.725842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.725903 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.726327 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.726328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.726329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.726475 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.726562 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.726916 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.727000 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:40:20.728231 29374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } attempt: 3
I20260806 06:40:20.728929 29539 catalog_manager.cc:1347] Generated new cluster ID: dd03d01655d84cf68d733c201385c330
I20260806 06:40:20.728983 29539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:40:20 dist-test-slave-3kk6 krb5kdc[29157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998420, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.189@KRBTEST.COM for kudu/127.29.142.188@KRBTEST.COM
I20260806 06:40:20.738175 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.728502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:42847 (local address 127.0.0.1:41764)
0806 06:40:20.728580 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:20.728590 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:20.728624 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:20.728918 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:20.728919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:20.729096 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:20.729135 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.729136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.730332 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.730333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.731158 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:20.731161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.731746 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.731748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:20.731820 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.732306 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:20.732311 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:20.733914 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:20.736818 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.736819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.736820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.737052 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.737345 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:20.737346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:20.737347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:20.737422 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:20.737831 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:20.737832 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:20.737901 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 06:40:20.737951 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:40:20.738057 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 06:40:20.739179 29488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.728701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41764 (local address 127.29.142.188:42847)
0806 06:40:20.728773 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.728775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.728782 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:40:20.728814 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.728815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.728824 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.728869 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.729300 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.730226 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.731270 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.731596 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.731677 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.734023 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.734024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.734025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.734032 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.736663 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.737149 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.737150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.737150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.737261 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.737507 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.737507 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.737507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.737744 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.738820 (+ 1076us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.738890 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.739020 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:40:20.740449 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:20.715523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53040 (local address 127.29.142.189:34039)
0806 06:40:20.715573 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.715575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.723379 (+ 7804us) server_negotiation.cc:407] Connection header received
0806 06:40:20.723494 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.723495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.723509 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.723543 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.724254 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.724979 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.725983 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:20.726278 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:20.726328 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:20.727872 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:20.727874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:20.727874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:20.727882 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:20.730823 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.739330 (+ 8507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.739332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.739332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.739394 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:20.739712 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:20.739713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:20.739713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:20.739854 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:20.739942 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:20.740268 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 06:40:20.740356 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:40:20.741110 29374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814: Bootstrap starting.
I20260806 06:40:20.741942 29374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:20.742270 29374 log.cc:824] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:20.743927 29374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814: No bootstrap required, opened a new log
I20260806 06:40:20.746086 29374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } }
I20260806 06:40:20.746243 29374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24fb5cfb03274726bdf7aab7e95b6814, State: Initialized, Role: FOLLOWER
I20260806 06:40:20.746325 29374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } }
I20260806 06:40:20.747036 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff37261ca0394b2da8f2840becb2b72b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "24fb5cfb03274726bdf7aab7e95b6814"
I20260806 06:40:20.747144 29341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:20.748095 29341 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff37261ca0394b2da8f2840becb2b72b in term 1.
I20260806 06:40:20.748598 29340 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [term 1 FOLLOWER]: Refusing update from remote peer ff37261ca0394b2da8f2840becb2b72b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:40:20.746970 29374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:20.749238 29536 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [LEADER]: Connected to new peer: Peer: permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:40:20.751883 29583 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:40:20.752000 29583 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:40:20.750945 29555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff37261ca0394b2da8f2840becb2b72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } } }
I20260806 06:40:20.752187 29555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:20.752277 29555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff37261ca0394b2da8f2840becb2b72b. Latest consensus state: current_term: 1 leader_uuid: "ff37261ca0394b2da8f2840becb2b72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } } }
I20260806 06:40:20.752383 29555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:20.752454 29555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ff37261ca0394b2da8f2840becb2b72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } } }
I20260806 06:40:20.752542 29555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:20.761447 29492 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ff37261ca0394b2da8f2840becb2b72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff37261ca0394b2da8f2840becb2b72b" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 37889 } } peers { permanent_uuid: "24fb5cfb03274726bdf7aab7e95b6814" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34039 } } peers { permanent_uuid: "e66c2241a2e94da8927d933f8ef52ddf" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 42847 } } }
I20260806 06:40:20.761600 29492 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:20.761850 29568 mvcc.cc:205] Tried to move back new op lower bound from 7315449531235864576 to 7315449531226185728. Current Snapshot: MvccSnapshot[applied={T|T < 7315449531235864576}]
I20260806 06:40:20.761966 29539 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:20.763715 29539 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:20.774758 29539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff37261ca0394b2da8f2840becb2b72b: Generated new TSK 0
I20260806 06:40:20.775017 29539 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:40:20.812604 29277 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:20.810594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52986 (local address 127.29.142.190:37889)
0806 06:40:20.810679 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.810681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.811827 (+ 1146us) server_negotiation.cc:407] Connection header received
0806 06:40:20.811946 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.811947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.811959 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.811993 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.812522 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52986: BlockingRecv error: recv got EOF from 127.0.0.1:52986 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 06:40:20.813467 29488 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:20.810905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41780 (local address 127.29.142.188:42847)
0806 06:40:20.810949 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.810951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.812851 (+ 1900us) server_negotiation.cc:407] Connection header received
0806 06:40:20.812963 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.812964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.812975 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.813006 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.813402 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41780: BlockingRecv error: recv got EOF from 127.0.0.1:41780 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 06:40:20.814297 29380 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:20.811363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53050 (local address 127.29.142.189:34039)
0806 06:40:20.811407 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.811409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.813736 (+ 2327us) server_negotiation.cc:407] Connection header received
0806 06:40:20.813841 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.813842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.813854 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.813884 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.814223 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53050: BlockingRecv error: recv got EOF from 127.0.0.1:53050 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 06:40:20.816902 29380 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:20.816069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53052 (local address 127.29.142.189:34039)
0806 06:40:20.816108 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.816110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.816349 (+ 239us) server_negotiation.cc:407] Connection header received
0806 06:40:20.816461 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.816463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.816482 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.816514 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.816856 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53052: BlockingRecv error: recv got EOF from 127.0.0.1:53052 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:40:20.819986 29380 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:20.819198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53066 (local address 127.29.142.189:34039)
0806 06:40:20.819231 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.819233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.819470 (+ 237us) server_negotiation.cc:407] Connection header received
0806 06:40:20.819569 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.819570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.819581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.819612 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.819943 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53066: BlockingRecv error: recv got EOF from 127.0.0.1:53066 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:40:20.824000 29380 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:20.823192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53072 (local address 127.29.142.189:34039)
0806 06:40:20.823237 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.823239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.823488 (+ 249us) server_negotiation.cc:407] Connection header received
0806 06:40:20.823588 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.823589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.823600 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.823630 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.823955 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53072: BlockingRecv error: recv got EOF from 127.0.0.1:53072 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:40:20.830192 29380 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:20.829177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53088 (local address 127.29.142.189:34039)
0806 06:40:20.829213 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.829215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.829465 (+ 250us) server_negotiation.cc:407] Connection header received
0806 06:40:20.829569 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.829570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.829582 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.829613 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.830142 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53088: BlockingRecv error: recv got EOF from 127.0.0.1:53088 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:40:20.840646 29380 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:20.839633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53102 (local address 127.29.142.189:34039)
0806 06:40:20.839688 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.839690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.840014 (+ 324us) server_negotiation.cc:407] Connection header received
0806 06:40:20.840121 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.840123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.840136 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.840167 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.840584 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53102: BlockingRecv error: recv got EOF from 127.0.0.1:53102 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 06:40:20.859119 29380 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:20.858131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53106 (local address 127.29.142.189:34039)
0806 06:40:20.858177 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.858180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.858473 (+ 293us) server_negotiation.cc:407] Connection header received
0806 06:40:20.858575 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.858577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.858590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.858620 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.859055 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53106: BlockingRecv error: recv got EOF from 127.0.0.1:53106 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:40:20.893482 29380 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:20.892578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.29.142.189:34039)
0806 06:40:20.892634 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.892636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.892811 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:40:20.892886 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.892887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.892900 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.892976 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.893402 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53110: BlockingRecv error: recv got EOF from 127.0.0.1:53110 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 06:40:20.959613 29380 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:20.958776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.29.142.189:34039)
0806 06:40:20.958852 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:20.958854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:20.958975 (+ 121us) server_negotiation.cc:407] Connection header received
0806 06:40:20.959022 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:20.959023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:20.959033 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:20.959099 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:20.959530 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53112: BlockingRecv error: recv got EOF from 127.0.0.1:53112 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 06:40:21.089600 29380 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:21.088843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53126 (local address 127.29.142.189:34039)
0806 06:40:21.089025 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:21.089027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:21.089039 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:40:21.089070 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:21.089071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:21.089083 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:21.089162 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:21.089507 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53126: BlockingRecv error: recv got EOF from 127.0.0.1:53126 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
W20260806 06:40:21.347785 29380 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:21.347044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53132 (local address 127.29.142.189:34039)
0806 06:40:21.347143 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:21.347145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:21.347157 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:40:21.347189 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:21.347190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:21.347216 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:21.347290 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:21.347664 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53132: BlockingRecv error: recv got EOF from 127.0.0.1:53132 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 06:40:21.700856 29528 catalog_manager.cc:1259] Loaded cluster ID: dd03d01655d84cf68d733c201385c330
I20260806 06:40:21.700966 29528 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf: loading cluster ID for follower catalog manager: success
I20260806 06:40:21.701812 29528 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf: acquiring CA information for follower catalog manager: success
I20260806 06:40:21.702183 29528 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e66c2241a2e94da8927d933f8ef52ddf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:40:21.752557 29583 catalog_manager.cc:1259] Loaded cluster ID: dd03d01655d84cf68d733c201385c330
I20260806 06:40:21.752653 29583 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814: loading cluster ID for follower catalog manager: success
I20260806 06:40:21.753525 29583 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814: acquiring CA information for follower catalog manager: success
I20260806 06:40:21.753878 29583 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 24fb5cfb03274726bdf7aab7e95b6814: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:40:21.863922 29683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:21.861113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53146 (local address 127.29.142.189:34039)
0806 06:40:21.861264 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:21.861267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:21.861369 (+ 102us) server_negotiation.cc:407] Connection header received
0806 06:40:21.861427 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:21.861428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:21.861519 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:21.861647 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:21.861840 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:21.862495 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:21.863257 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:21.863444 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:21.863498 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:21.863644 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:21.863750 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:21.863810 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:21.863839 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 06:40:21.863855 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260806 06:40:21.868132 29691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:21.865824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41794 (local address 127.29.142.188:42847)
0806 06:40:21.865942 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:21.865944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:21.865976 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:40:21.866043 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:21.866043 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:21.866077 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:21.866144 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:21.866250 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:21.866789 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:21.867447 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:21.867629 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:21.867676 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:21.867814 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:21.867907 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:21.867964 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:21.867998 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 06:40:21.868014 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":51,"threads_started":1}
I20260806 06:40:21.872892 29691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:21.869860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.29.142.188:42847)
0806 06:40:21.869901 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:21.869903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:21.870018 (+ 115us) server_negotiation.cc:407] Connection header received
0806 06:40:21.870115 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:21.870116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:21.870127 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:21.870156 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:21.870369 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:21.870986 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:21.871926 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:21.872220 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:21.872290 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:21.872460 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:21.872495 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:21.872553 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:21.872793 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 06:40:21.872813 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:40:21.873126 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:21.869769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52994 (local address 127.29.142.190:37889)
0806 06:40:21.869873 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:21.869875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:21.870009 (+ 134us) server_negotiation.cc:407] Connection header received
0806 06:40:21.870093 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:21.870095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:21.870146 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:21.870227 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:21.870414 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:21.871253 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:21.872188 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:21.872490 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:21.872563 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:21.872762 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:21.872874 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:21.872940 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:21.872988 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 06:40:21.873020 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":47,"threads_started":1}
I20260806 06:40:21.873394 29683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:21.870065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53152 (local address 127.29.142.189:34039)
0806 06:40:21.870109 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:21.870111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:21.870313 (+ 202us) server_negotiation.cc:407] Connection header received
0806 06:40:21.870419 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:21.870421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:21.870434 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:21.870488 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:21.870634 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:21.871358 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:21.872269 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:21.872577 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:21.872639 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:21.873116 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:21.873148 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:21.873198 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:21.873293 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 06:40:21.873322 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:40:21.875226 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 29174
I20260806 06:40:21.887342 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 29282
I20260806 06:40:21.899166 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 29384
2026-08-06T06:40:21Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3081 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:40:21 dist-test-slave-3kk6 krb5kdc[29708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:40:21 dist-test-slave-3kk6 krb5kdc[29708](info): set up 2 sockets
Aug 06 06:40:21 dist-test-slave-3kk6 krb5kdc[29708](info): commencing operation
krb5kdc: starting...
W20260806 06:40:23.114950 30266 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/test-user.keytab.
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:40:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:40:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.142.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
Entry for principal kudu/127.29.142.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab.
I20260806 06:40:23.253398 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:36863
--webserver_interface=127.29.142.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:36863,127.29.142.189:34327,127.29.142.188:46141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:40:23.341707 29725 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:40:23.341939 29725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:23.341958 29725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:23.341979 29725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:23.343218 29725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:23.343263 29725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:23.343279 29725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:23.343294 29725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:23.343346 29725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:23.344861 29725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:36863,127.29.142.189:34327,127.29.142.188:46141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.190.keytab
--principal=kudu/127.29.142.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.190:36863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.142.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:23.345132 29725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:23.345386 29725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:23.350345 29732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:23.350387 29731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:23.350363 29734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:23.350798 29725 server_base.cc:1034] running on GCE node
I20260806 06:40:23.351061 29725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:23.351375 29725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:23.352555 29725 hybrid_clock.cc:648] HybridClock initialized: now 1785998423352517 us; error 59 us; skew 500 ppm
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:23.355389 29725 init.cc:377] Logged in from keytab as kudu/127.29.142.190@KRBTEST.COM (short username kudu)
I20260806 06:40:23.355914 29725 webserver.cc:460] Webserver started at http://127.29.142.190:42255/ using document root <none> and password file <none>
I20260806 06:40:23.356165 29725 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:23.356237 29725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:23.356397 29725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:23.357539 29725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/data/instance:
uuid: "3afb55b90f294c0c82e59798422b06ea"
format_stamp: "Formatted at 2026-08-06 06:40:23 on dist-test-slave-3kk6"
server_key: "cabb84fe832030ef7b7481c6570266f3"
server_key_iv: "8b4350c7a183418da1544e1900f43e11"
server_key_version: "encryptionkey@0"
I20260806 06:40:23.357929 29725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/wal/instance:
uuid: "3afb55b90f294c0c82e59798422b06ea"
format_stamp: "Formatted at 2026-08-06 06:40:23 on dist-test-slave-3kk6"
server_key: "cabb84fe832030ef7b7481c6570266f3"
server_key_iv: "8b4350c7a183418da1544e1900f43e11"
server_key_version: "encryptionkey@0"
I20260806 06:40:23.359438 29725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:40:23.360205 29744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:23.360395 29725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:40:23.360512 29725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/wal
uuid: "3afb55b90f294c0c82e59798422b06ea"
format_stamp: "Formatted at 2026-08-06 06:40:23 on dist-test-slave-3kk6"
server_key: "cabb84fe832030ef7b7481c6570266f3"
server_key_iv: "8b4350c7a183418da1544e1900f43e11"
server_key_version: "encryptionkey@0"
I20260806 06:40:23.360591 29725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:23.374547 29725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:23.377380 29725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:23.377578 29725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:23.382423 29725 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.190:36863
I20260806 06:40:23.382462 29817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.190:36863 every 8 connection(s)
I20260806 06:40:23.382915 29725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/data/info.pb
I20260806 06:40:23.384002 29818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:23.385584 29818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:23.389770 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 29725
I20260806 06:40:23.389859 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-0/wal/instance
I20260806 06:40:23.390082 30266 external_mini_cluster.cc:1428] Setting key e091aed4a90a1ac5515eabec7d284cd9
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:40:23.396512 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.386278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:36863 (local address 127.0.0.1:46922)
0806 06:40:23.386451 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.386490 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:23.386993 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:23.387279 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:23.387282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:23.387537 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:23.387698 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.387702 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.388758 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.388759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.389436 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.389437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.390030 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.390031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.390116 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.390901 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:23.390912 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:23.392640 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:23.394720 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.394729 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.394732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.395040 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.395362 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.395367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.395367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.395455 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.396000 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:23.396007 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:23.396075 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 06:40:23.396162 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:40:23.396318 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260806 06:40:23.396613 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.386451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46922 (local address 127.29.142.190:36863)
0806 06:40:23.386587 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.386592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.386633 (+ 41us) server_negotiation.cc:407] Connection header received
0806 06:40:23.387135 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.387137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.387154 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.387247 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.387816 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.388638 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.389518 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.389873 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.389979 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.392735 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.392743 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.392743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.392751 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.394574 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.395133 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.395136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.395136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.395260 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.395542 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.395543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.395544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.395889 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.395996 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.396259 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.396421 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260806 06:40:23.397579 29818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:23.398039 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.397878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34327 (local address 127.0.0.1:35404)
0806 06:40:23.397945 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.398006 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 06:40:23.398182 29746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34327: connect: Connection refused (error 111)
I20260806 06:40:23.398689 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.398557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34327 (local address 127.0.0.1:35420)
0806 06:40:23.398607 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.398636 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:40:23.398818 29818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.189:34327: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34327: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.29.142.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.189.keytab.
Entry for principal kudu/127.29.142.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.189.keytab.
I20260806 06:40:23.417686 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.189:34327
--webserver_interface=127.29.142.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.189.keytab
--principal=kudu/127.29.142.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:36863,127.29.142.189:34327,127.29.142.188:46141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:40:23.448074 29818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } attempt: 1
I20260806 06:40:23.448691 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.448503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34327 (local address 127.0.0.1:35430)
0806 06:40:23.448585 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.448646 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 06:40:23.449193 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.449082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34327 (local address 127.0.0.1:35438)
0806 06:40:23.449136 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.449171 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 06:40:23.449343 29818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.189:34327: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34327: connect: Connection refused (error 111)
W20260806 06:40:23.502738 29828 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:40:23.502964 29828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:23.502982 29828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:23.503052 29828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:23.504232 29828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:23.504280 29828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:23.504295 29828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:23.504307 29828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:23.504320 29828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:23.505844 29828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:36863,127.29.142.189:34327,127.29.142.188:46141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.189.keytab
--principal=kudu/127.29.142.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.189:34327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.142.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:23.506116 29828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:23.506362 29828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:23.511511 29835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:23.511545 29836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:23.511711 29828 server_base.cc:1034] running on GCE node
W20260806 06:40:23.511520 29840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:23.512130 29828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:23.512461 29828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:23.513638 29828 hybrid_clock.cc:648] HybridClock initialized: now 1785998423513635 us; error 42 us; skew 500 ppm
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:23.516386 29828 init.cc:377] Logged in from keytab as kudu/127.29.142.189@KRBTEST.COM (short username kudu)
I20260806 06:40:23.516902 29828 webserver.cc:460] Webserver started at http://127.29.142.189:38741/ using document root <none> and password file <none>
I20260806 06:40:23.517138 29828 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:23.517208 29828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:23.517372 29828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:23.517524 29818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } attempt: 2
I20260806 06:40:23.517956 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.517825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34327 (local address 127.0.0.1:35454)
0806 06:40:23.517889 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.517927 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 06:40:23.518342 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.518229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34327 (local address 127.0.0.1:35462)
0806 06:40:23.518286 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.518321 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 06:40:23.518440 29818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.189:34327: Network error: Client connection negotiation failed: client connection to 127.29.142.189:34327: connect: Connection refused (error 111)
I20260806 06:40:23.518827 29828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/data/instance:
uuid: "2ae7491358ff45ffb9692d43678b05e3"
format_stamp: "Formatted at 2026-08-06 06:40:23 on dist-test-slave-3kk6"
server_key: "b59b1d6de611917cbd93025a034d7cee"
server_key_iv: "953d46de172f5c820b4cfab0c78b9aad"
server_key_version: "encryptionkey@0"
I20260806 06:40:23.519208 29828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/wal/instance:
uuid: "2ae7491358ff45ffb9692d43678b05e3"
format_stamp: "Formatted at 2026-08-06 06:40:23 on dist-test-slave-3kk6"
server_key: "b59b1d6de611917cbd93025a034d7cee"
server_key_iv: "953d46de172f5c820b4cfab0c78b9aad"
server_key_version: "encryptionkey@0"
I20260806 06:40:23.520607 29828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:40:23.521374 29853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:23.521565 29828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:40:23.521667 29828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/wal
uuid: "2ae7491358ff45ffb9692d43678b05e3"
format_stamp: "Formatted at 2026-08-06 06:40:23 on dist-test-slave-3kk6"
server_key: "b59b1d6de611917cbd93025a034d7cee"
server_key_iv: "953d46de172f5c820b4cfab0c78b9aad"
server_key_version: "encryptionkey@0"
I20260806 06:40:23.521749 29828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:23.531710 29828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:23.533794 29828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:23.533962 29828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:23.538383 29828 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.189:34327
I20260806 06:40:23.538416 29924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.189:34327 every 8 connection(s)
I20260806 06:40:23.538967 29828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/data/info.pb
I20260806 06:40:23.539953 29925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:23.541528 29925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:23.543522 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 29828
I20260806 06:40:23.543649 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-1/wal/instance
I20260806 06:40:23.543853 30266 external_mini_cluster.cc:1428] Setting key 9fb13747cc3bbb5697b92870296756c4
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.189@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:40:23.551849 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.542363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46936 (local address 127.29.142.190:36863)
0806 06:40:23.542449 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.542451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.542712 (+ 261us) server_negotiation.cc:407] Connection header received
0806 06:40:23.543230 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.543232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.543245 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.543306 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.544263 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.544759 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.546062 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.546274 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.546368 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.548744 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.548748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.548748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.548760 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.550285 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.550751 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.550753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.550753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.550820 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.551098 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.551099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.551099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.551229 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.551304 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.551595 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.551747 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 06:40:23.551987 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.542317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:36863 (local address 127.0.0.1:46936)
0806 06:40:23.542536 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.542575 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:23.543119 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:23.543427 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:23.543432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:23.543833 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:23.544140 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.544146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.544860 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.544861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.545955 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.545958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.546377 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.546378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.546505 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.547221 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:23.547230 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:23.548619 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:23.550389 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.550396 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.550399 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.550632 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.550900 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.550901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.550901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.551019 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.551309 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:23.551316 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:23.551458 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 06:40:23.551559 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 06:40:23.551746 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.29.142.188@KRBTEST.COM; defaulting to no policy
I20260806 06:40:23.552567 29925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.142.188@KRBTEST.COM" created.
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.189@KRBTEST.COM for kudu/127.29.142.189@KRBTEST.COM
I20260806 06:40:23.560802 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.552858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34327 (local address 127.0.0.1:35478)
0806 06:40:23.552910 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.552921 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:23.552963 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:23.553287 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:23.553288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:23.553408 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:23.553462 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.553463 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.554384 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.554385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.555098 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.555101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.555632 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.555633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.555694 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.556177 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:23.556182 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:23.557494 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:23.559619 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.559620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.559621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.559805 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.560047 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.560048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.560048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.560105 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.560421 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:23.560422 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:23.560464 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:40:23.560548 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:40:23.560664 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:40:23.560825 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.552947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.29.142.189:34327)
0806 06:40:23.553141 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.553145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.553161 (+ 16us) server_negotiation.cc:407] Connection header received
0806 06:40:23.553182 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.553184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.553195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.553250 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.553534 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.554215 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.555219 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.555536 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.555632 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.557602 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.557613 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.557613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.557625 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.559494 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.559905 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.559908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.559909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.559970 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.560168 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.560169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.560169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.560337 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.560433 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.560583 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.560720 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":170,"thread_start_us":141,"threads_started":1}
I20260806 06:40:23.561511 29925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:23.561939 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.561811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44738)
0806 06:40:23.561866 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.561915 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 06:40:23.562139 29854 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
I20260806 06:40:23.562777 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.562569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44752)
0806 06:40:23.562644 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.562746 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 06:40:23.562951 29925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:46141: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.142.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.188.keytab.
Entry for principal kudu/127.29.142.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.188.keytab.
I20260806 06:40:23.571659 30266 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrotugL/build/release/bin/kudu
/tmp/dist-test-taskrotugL/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.188:46141
--webserver_interface=127.29.142.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.188.keytab
--principal=kudu/127.29.142.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.142.148:41525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.142.190:36863,127.29.142.189:34327,127.29.142.188:46141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:40:23.609602 29818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } attempt: 3
I20260806 06:40:23.612205 29925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } attempt: 1
I20260806 06:40:23.612665 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.612504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44764)
0806 06:40:23.612578 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.612627 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 06:40:23.613022 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.612920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44772)
0806 06:40:23.612973 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.613003 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 06:40:23.613108 29925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:46141: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for kudu/127.29.142.189@KRBTEST.COM
I20260806 06:40:23.617611 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.610163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34327 (local address 127.0.0.1:35480)
0806 06:40:23.610249 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.610262 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:23.610324 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:23.610493 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:23.610494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:23.610726 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:23.610806 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.610809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.611489 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.611491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.612372 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.612375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.612885 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.612886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.612976 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.613501 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:23.613506 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:23.614869 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:23.616514 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.616515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.616516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.616699 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.616920 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.616921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.616922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.616984 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.617266 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:23.617267 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:23.617317 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 06:40:23.617398 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:40:23.617509 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 06:40:23.617609 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.610275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35480 (local address 127.29.142.189:34327)
0806 06:40:23.610328 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.610331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.610346 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:40:23.610384 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.610384 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.610395 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.610458 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.610893 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.611403 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.612460 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.612754 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.612822 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.614963 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.614965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.614965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.614973 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.616428 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.616794 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.616795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.616795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.616855 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.617071 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.617072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.617073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.617189 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.617262 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.617439 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.617516 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:40:23.617980 29818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:23.618360 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.618250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44778)
0806 06:40:23.618305 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.618333 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:40:23.618724 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.618608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44780)
0806 06:40:23.618669 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.618693 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:40:23.618847 29818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:46141: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
I20260806 06:40:23.650015 29818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } attempt: 1
I20260806 06:40:23.650767 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.650492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44796)
0806 06:40:23.650585 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.650636 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 06:40:23.651217 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.651106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44798)
0806 06:40:23.651159 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.651198 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 06:40:23.651356 29818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:46141: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
W20260806 06:40:23.658716 29935 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:40:23.658977 29935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:40:23.658995 29935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:40:23.659018 29935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:40:23.660184 29935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:40:23.660223 29935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:40:23.660238 29935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:40:23.660251 29935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:40:23.660264 29935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:40:23.661707 29935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.142.148:41525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.142.190:36863,127.29.142.189:34327,127.29.142.188:46141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/krb5kdc/kudu_127.29.142.188.keytab
--principal=kudu/127.29.142.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.142.188:46141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.142.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:11:02 UTC on ccycloud.quasar-hvzbbz.root.comops.site
build id 4569617
I20260806 06:40:23.661914 29935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:40:23.662097 29935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:40:23.667145 29949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:23.667173 29946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:40:23.667178 29945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:40:23.667351 29935 server_base.cc:1034] running on GCE node
I20260806 06:40:23.667590 29935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:40:23.667918 29935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:40:23.669095 29935 hybrid_clock.cc:648] HybridClock initialized: now 1785998423669077 us; error 38 us; skew 500 ppm
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:40:23.671851 29935 init.cc:377] Logged in from keytab as kudu/127.29.142.188@KRBTEST.COM (short username kudu)
I20260806 06:40:23.672370 29935 webserver.cc:460] Webserver started at http://127.29.142.188:38001/ using document root <none> and password file <none>
I20260806 06:40:23.672634 29935 fs_manager.cc:359] Metadata directory not provided
I20260806 06:40:23.672708 29935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:40:23.672864 29935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:40:23.674001 29935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/data/instance:
uuid: "15c304e120944f8f874f0197dca0522b"
format_stamp: "Formatted at 2026-08-06 06:40:23 on dist-test-slave-3kk6"
server_key: "47e4b9a987516390bc9cbccdc0ff31b8"
server_key_iv: "b559f11385698f009848d6c299390ef3"
server_key_version: "encryptionkey@0"
I20260806 06:40:23.674393 29935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/wal/instance:
uuid: "15c304e120944f8f874f0197dca0522b"
format_stamp: "Formatted at 2026-08-06 06:40:23 on dist-test-slave-3kk6"
server_key: "47e4b9a987516390bc9cbccdc0ff31b8"
server_key_iv: "b559f11385698f009848d6c299390ef3"
server_key_version: "encryptionkey@0"
I20260806 06:40:23.675938 29935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:40:23.676759 29959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:40:23.676942 29935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:40:23.677047 29935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/data,/tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/wal
uuid: "15c304e120944f8f874f0197dca0522b"
format_stamp: "Formatted at 2026-08-06 06:40:23 on dist-test-slave-3kk6"
server_key: "47e4b9a987516390bc9cbccdc0ff31b8"
server_key_iv: "b559f11385698f009848d6c299390ef3"
server_key_version: "encryptionkey@0"
I20260806 06:40:23.677142 29935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:40:23.681599 29925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } attempt: 2
I20260806 06:40:23.682080 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.681917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44812)
0806 06:40:23.681994 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.682040 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 06:40:23.682513 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.682426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44822)
0806 06:40:23.682470 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.682496 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:40:23.682611 29925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.142.188:46141: Network error: Client connection negotiation failed: client connection to 127.29.142.188:46141: connect: Connection refused (error 111)
I20260806 06:40:23.688524 29935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:40:23.690632 29935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:40:23.690832 29935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:40:23.695127 29935 rpc_server.cc:307] RPC server started. Bound to: 127.29.142.188:46141
I20260806 06:40:23.695168 30033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.142.188:46141 every 8 connection(s)
I20260806 06:40:23.695588 29935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/data/info.pb
I20260806 06:40:23.696537 30034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:40:23.697454 30266 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrotugL/build/release/bin/kudu as pid 29935
I20260806 06:40:23.697573 30266 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrotugL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998137760399-30266-0/minicluster-data/master-2/wal/instance
I20260806 06:40:23.697793 30266 external_mini_cluster.cc:1428] Setting key 6dce9383ad7b49ba96b696e7ead51b92
I20260806 06:40:23.698868 30034 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.188@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.188@KRBTEST.COM
I20260806 06:40:23.708387 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.699933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46948 (local address 127.29.142.190:36863)
0806 06:40:23.699991 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.699993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.700115 (+ 122us) server_negotiation.cc:407] Connection header received
0806 06:40:23.700194 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.700195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.700205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.700246 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.700894 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.701382 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.702301 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.702499 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.702609 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.704973 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.704975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.704975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.704984 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.706796 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.707223 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.707224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.707225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.707292 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.707589 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.707590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.707590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.707730 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.707813 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.708187 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.708261 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:40:23.708560 30041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.699851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.190:36863 (local address 127.0.0.1:46948)
0806 06:40:23.700018 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.700057 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:23.700130 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:23.700288 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:23.700291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:23.700647 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:23.700761 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.700767 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.701484 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.701485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.702207 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.702209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.702631 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.702632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.702725 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.703401 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:23.703418 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:23.704874 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:23.706879 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.706885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.706887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.707144 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.707365 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.707365 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.707365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.707509 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.707842 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:23.707847 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:23.708013 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 06:40:23.708144 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 06:40:23.708316 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
I20260806 06:40:23.709079 30040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.698384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44836 (local address 127.29.142.188:46141)
0806 06:40:23.698571 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.698580 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.698607 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:40:23.699105 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.699112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.699162 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.699280 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.700171 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.701311 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.702118 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.702502 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.702628 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.705491 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.705498 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.705498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.705528 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.707400 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.707805 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.707808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.707809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.707868 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.708065 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.708066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.708066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.708244 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.708374 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.708865 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.708963 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260806 06:40:23.709192 30034 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.188@KRBTEST.COM for kudu/127.29.142.189@KRBTEST.COM
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.190@KRBTEST.COM
I20260806 06:40:23.717741 30041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.709473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.189:34327 (local address 127.0.0.1:35496)
0806 06:40:23.709619 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.709628 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:23.709665 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:23.709882 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:23.709883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:23.710019 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:23.710069 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.710070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.710753 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.710754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.711444 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.711446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.712094 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.712095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.712175 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.713254 (+ 1079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:23.713259 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:23.714778 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:23.716610 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.716612 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.716612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.716898 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.717114 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.717115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.717115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.717216 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.717466 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:23.717467 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:23.717517 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 06:40:23.717592 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:40:23.717671 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260806 06:40:23.717859 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.709545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35496 (local address 127.29.142.189:34327)
0806 06:40:23.709595 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.709597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.709754 (+ 157us) server_negotiation.cc:407] Connection header received
0806 06:40:23.709785 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.709786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.709798 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.709864 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.710158 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.710633 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.711559 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.711964 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.712012 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.714866 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.714867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.714868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.714875 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.716503 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.716990 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.716991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.716991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.717045 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.717285 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.717286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.717286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.717397 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.717448 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.717646 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.717728 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:40:23.718348 30034 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:40:23.718775 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.709946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46960 (local address 127.29.142.190:36863)
0806 06:40:23.709986 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.709988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.709995 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:40:23.710017 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.710017 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.710026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.710075 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.710642 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.711320 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.711995 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.712223 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.712300 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.715533 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.715535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.715535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.715543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.717446 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.717847 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.717848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.717848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.717911 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.718143 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.718144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.718145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.718283 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.718511 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.718560 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.718649 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.188@KRBTEST.COM for kudu/127.29.142.188@KRBTEST.COM
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.142.189@KRBTEST.COM
I20260806 06:40:23.725436 30041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.718629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44838)
0806 06:40:23.718697 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.718710 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:23.718749 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:23.718915 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:23.718915 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:23.719080 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:23.719113 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.719115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.719957 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.719958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.720422 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.720424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.720833 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.720833 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.720881 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.721316 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:23.721321 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:23.722525 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:23.724248 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.724249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.724249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.724464 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.724691 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.724692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.724692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.724752 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.725033 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:23.725033 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:23.725071 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 06:40:23.725158 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:40:23.725282 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 06:40:23.725502 30040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.718732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44838 (local address 127.29.142.188:46141)
0806 06:40:23.718770 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.718771 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.718780 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:40:23.718802 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.718802 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.718813 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.718876 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.719208 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.719882 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.720514 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.720758 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.720836 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.722613 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.722615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.722615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.722621 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.724105 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.724554 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.724555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.724556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.724612 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.724837 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.724837 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.724838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.724966 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.725028 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.725206 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.725335 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":20}
I20260806 06:40:23.726518 29818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } attempt: 2
I20260806 06:40:23.727838 30034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b: Bootstrap starting.
I20260806 06:40:23.727910 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.719597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35506 (local address 127.29.142.189:34327)
0806 06:40:23.719641 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.719643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.720019 (+ 376us) server_negotiation.cc:407] Connection header received
0806 06:40:23.720078 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.720079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.720089 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.720138 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.720609 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.721054 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.721721 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.721934 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.722021 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.724718 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.724719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.724720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.724727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.726569 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.727082 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.727083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.727083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.727149 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.727366 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.727367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.727368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.727482 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.727676 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.727716 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.727793 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:40:23.729357 30034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:23.729493 30266 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:40:23.730038 30034 log.cc:824] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:23.731060 30034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b: No bootstrap required, opened a new log
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.190@KRBTEST.COM for kudu/127.29.142.188@KRBTEST.COM
I20260806 06:40:23.730580 29780 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3afb55b90f294c0c82e59798422b06ea"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46960
I20260806 06:40:23.733203 30034 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } }
I20260806 06:40:23.733335 30034 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c304e120944f8f874f0197dca0522b, State: Initialized, Role: FOLLOWER
I20260806 06:40:23.733446 30034 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } }
I20260806 06:40:23.733829 30034 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:23.734091 30044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } } }
I20260806 06:40:23.734187 30044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:23.734896 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.726814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44844)
0806 06:40:23.726857 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.726868 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:23.726907 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:23.727065 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:23.727066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:23.727240 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:23.727293 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.727295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.728166 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.728167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.728860 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.728862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.729422 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.729423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.729485 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.729945 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:23.729951 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:23.731747 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:23.733665 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.733666 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.733667 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.733910 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.734111 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.734127 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.734127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.734195 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.734527 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:23.734528 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:23.734590 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 06:40:23.734687 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 06:40:23.734797 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 06:40:23.734918 30040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.726909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.29.142.188:46141)
0806 06:40:23.726949 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.726951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.726959 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:40:23.726978 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.726979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.726987 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.727370 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.727402 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.728051 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.728953 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.729248 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.729323 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.731858 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.731859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.731860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.731867 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.733551 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.733990 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.733991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.733992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.734050 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.734269 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.734269 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.734270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.734434 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.734522 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.734711 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.734836 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 06:40:23.735973 30057 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:40:23.736034 30057 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:40:23.736964 29818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea: Bootstrap starting.
I20260806 06:40:23.737807 29818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:23.738121 29818 log.cc:824] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:23.739034 29818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea: No bootstrap required, opened a new log
I20260806 06:40:23.736224 29935 master_runner.cc:186] Error getting master registration for 127.29.142.190:36863: OK, instance_id { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" instance_seqno: 1785998423381044 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:40:23.739493 29935 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:40:23.740187 29818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } }
I20260806 06:40:23.740301 29818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3afb55b90f294c0c82e59798422b06ea, State: Initialized, Role: FOLLOWER
I20260806 06:40:23.740387 29818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } }
I20260806 06:40:23.740670 30061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } } }
I20260806 06:40:23.740749 29818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:23.740756 30061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:23.742075 29780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 06:40:23.742146 29780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [term 0 FOLLOWER]: Advancing to term 1
W20260806 06:40:23.742507 30078 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:40:23.742571 30078 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:40:23.743040 29780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } }
I20260806 06:40:23.743340 29780 leader_election.cc:288] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [CANDIDATE]: Term 1 election: Requested vote from peers 2ae7491358ff45ffb9692d43678b05e3 (127.29.142.189:34327), 15c304e120944f8f874f0197dca0522b (127.29.142.188:46141)
I20260806 06:40:23.743935 29996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3afb55b90f294c0c82e59798422b06ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "15c304e120944f8f874f0197dca0522b"
I20260806 06:40:23.744058 29996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:23.745101 29996 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3afb55b90f294c0c82e59798422b06ea in term 1.
I20260806 06:40:23.745373 29745 leader_election.cc:302] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c304e120944f8f874f0197dca0522b, 3afb55b90f294c0c82e59798422b06ea; no voters:
I20260806 06:40:23.745544 30061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:40:23.745698 30061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [term 1 LEADER]: Becoming Leader. State: Replica: 3afb55b90f294c0c82e59798422b06ea, State: Running, Role: LEADER
I20260806 06:40:23.745793 30061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } }
I20260806 06:40:23.746209 30061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3afb55b90f294c0c82e59798422b06ea. Latest consensus state: current_term: 1 leader_uuid: "3afb55b90f294c0c82e59798422b06ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } } }
I20260806 06:40:23.746290 30061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:23.746542 29725 master_runner.cc:186] Error getting master registration for 127.29.142.189:34327: OK, instance_id { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" instance_seqno: 1785998423537129 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:40:23.748950 29725 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:40:23.755201 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.744758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35518 (local address 127.29.142.189:34327)
0806 06:40:23.744811 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.744813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.745622 (+ 809us) server_negotiation.cc:407] Connection header received
0806 06:40:23.745747 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.745748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.745760 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.745794 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.746633 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.747428 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.748426 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.749299 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.749350 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.751424 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.751425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.751425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.751433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.753770 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.754229 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.754231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.754231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.754296 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.754520 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.754521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.754521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.754700 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.754778 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.755011 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.755107 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:40:23.755434 30091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:40:23.755662 30091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:40:23.755808 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.745170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46974 (local address 127.29.142.190:36863)
0806 06:40:23.745211 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.745212 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.745981 (+ 769us) server_negotiation.cc:407] Connection header received
0806 06:40:23.746149 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.746150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.746161 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.746204 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.747369 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.748058 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.749647 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.749947 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.749990 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.752331 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.752332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.752333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.752341 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.754019 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.754687 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.754689 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.754689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.754747 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.755066 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.755068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.755068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.755217 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.755306 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.755647 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.755729 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:40:23.758085 29996 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [term 1 FOLLOWER]: Refusing update from remote peer 3afb55b90f294c0c82e59798422b06ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:40:23.758518 30089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:40:23.760460 30044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3afb55b90f294c0c82e59798422b06ea. Latest consensus state: current_term: 1 leader_uuid: "3afb55b90f294c0c82e59798422b06ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } } }
I20260806 06:40:23.760588 30044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:23.769106 30040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.746448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44846 (local address 127.29.142.188:46141)
0806 06:40:23.746524 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.746526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.756027 (+ 9501us) server_negotiation.cc:407] Connection header received
0806 06:40:23.756133 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.756135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.756148 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.756181 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.757042 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.757755 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.758851 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.759113 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.759159 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.766157 (+ 6998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.766158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.766159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.766167 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.767670 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.768117 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.768160 (+ 43us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.768160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.768229 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.768503 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.768504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.768504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.768648 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.768740 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.768908 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.768986 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 06:40:23.770961 30061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3afb55b90f294c0c82e59798422b06ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } } }
I20260806 06:40:23.771088 30061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:23.771167 30092 mvcc.cc:205] Tried to move back new op lower bound from 7315449543709351936 to 7315449543663419392. Current Snapshot: MvccSnapshot[applied={T|T < 7315449543709351936}]
I20260806 06:40:23.771178 30044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3afb55b90f294c0c82e59798422b06ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } } }
I20260806 06:40:23.771271 30044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:23.771485 30093 mvcc.cc:205] Tried to move back new op lower bound from 7315449543709351936 to 7315449543663419392. Current Snapshot: MvccSnapshot[applied={T|T < 7315449543709351936}]
I20260806 06:40:23.771870 30091 catalog_manager.cc:1347] Generated new cluster ID: 7e4c7a3905224ccf98c9b75bb59adf8d
I20260806 06:40:23.771922 30091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:40:23.773737 29925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } attempt: 3
Aug 06 06:40:23 dist-test-slave-3kk6 krb5kdc[29708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998423, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.142.189@KRBTEST.COM for kudu/127.29.142.188@KRBTEST.COM
I20260806 06:40:23.781953 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.774055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.142.188:46141 (local address 127.0.0.1:44862)
0806 06:40:23.774110 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:40:23.774123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:40:23.774162 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:40:23.774306 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:40:23.774307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:40:23.774499 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:40:23.774553 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.774555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.775448 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.775449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.776257 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:40:23.776261 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.776704 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.776705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:40:23.776770 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.777278 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:40:23.777284 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:40:23.778916 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:40:23.780815 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.780829 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.780830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.781052 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.781298 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:40:23.781299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:40:23.781299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:40:23.781367 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:40:23.781636 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:40:23.781637 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:40:23.781693 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 06:40:23.781753 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:40:23.781846 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:40:23.781965 30040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.774117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44862 (local address 127.29.142.188:46141)
0806 06:40:23.774166 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.774168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.774219 (+ 51us) server_negotiation.cc:407] Connection header received
0806 06:40:23.774240 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.774241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.774250 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.774297 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.774620 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.775378 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.776339 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.776631 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.776689 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.779025 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:40:23.779026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:40:23.779026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:40:23.779034 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:40:23.780723 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.781123 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.781124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.781124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.781187 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:40:23.781412 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:40:23.781413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:40:23.781413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:40:23.781552 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:40:23.781628 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.781778 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.781864 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:40:23.783927 29925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3: Bootstrap starting.
I20260806 06:40:23.784760 29925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3: Neither blocks nor log segments found. Creating new log.
I20260806 06:40:23.785077 29925 log.cc:824] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:40:23.788811 30091 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:40:23.789579 30091 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:40:23.799572 29925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3: No bootstrap required, opened a new log
I20260806 06:40:23.801764 29925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } }
I20260806 06:40:23.801955 29925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ae7491358ff45ffb9692d43678b05e3, State: Initialized, Role: FOLLOWER
I20260806 06:40:23.802058 29925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } }
I20260806 06:40:23.802467 29925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:40:23.802615 29886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:40:23.804312 29886 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [term 1 FOLLOWER]: Refusing update from remote peer 3afb55b90f294c0c82e59798422b06ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:40:23.804471 30100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } } }
I20260806 06:40:23.804697 30089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:40:23.804574 30100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:23.807418 30100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3afb55b90f294c0c82e59798422b06ea. Latest consensus state: current_term: 1 leader_uuid: "3afb55b90f294c0c82e59798422b06ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } } }
I20260806 06:40:23.807541 30100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:23.807590 30100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3afb55b90f294c0c82e59798422b06ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } } }
I20260806 06:40:23.807642 30100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:40:23.803010 29888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3afb55b90f294c0c82e59798422b06ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2ae7491358ff45ffb9692d43678b05e3"
I20260806 06:40:23.809386 29888 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3afb55b90f294c0c82e59798422b06ea for term 1 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260806 06:40:23.809727 30091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea: Generated new TSK 0
I20260806 06:40:23.809772 30119 mvcc.cc:205] Tried to move back new op lower bound from 7315449543895228416 to 7315449543663419392. Current Snapshot: MvccSnapshot[applied={T|T < 7315449543709351936}]
I20260806 06:40:23.809943 30091 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:40:23.811704 30117 catalog_manager.cc:1259] Loaded cluster ID: 7e4c7a3905224ccf98c9b75bb59adf8d
I20260806 06:40:23.811758 30117 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3: loading cluster ID for follower catalog manager: success
I20260806 06:40:23.812901 30117 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3: acquiring CA information for follower catalog manager: success
I20260806 06:40:23.813220 30117 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2ae7491358ff45ffb9692d43678b05e3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 06:40:23.841383 29929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:23.839657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35526 (local address 127.29.142.189:34327)
0806 06:40:23.839717 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.839720 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.840653 (+ 933us) server_negotiation.cc:407] Connection header received
0806 06:40:23.840779 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.840780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.840792 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.840824 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.841309 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35526: BlockingRecv error: recv got EOF from 127.0.0.1:35526 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 06:40:23.842173 29822 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:23.840094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46990 (local address 127.29.142.190:36863)
0806 06:40:23.840136 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.840137 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.841629 (+ 1492us) server_negotiation.cc:407] Connection header received
0806 06:40:23.841745 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.841746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.841758 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.841787 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.842110 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46990: BlockingRecv error: recv got EOF from 127.0.0.1:46990 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:40:23.842926 30040 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:23.840404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44876 (local address 127.29.142.188:46141)
0806 06:40:23.840463 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.840465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.842401 (+ 1936us) server_negotiation.cc:407] Connection header received
0806 06:40:23.842497 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.842498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.842509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.842538 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.842866 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44876: BlockingRecv error: recv got EOF from 127.0.0.1:44876 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:40:23.848416 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.844569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35538 (local address 127.29.142.189:34327)
0806 06:40:23.844607 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.844609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.844864 (+ 255us) server_negotiation.cc:407] Connection header received
0806 06:40:23.844975 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.844976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.845026 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.845071 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:23.845281 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.846072 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.847372 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.847705 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.847752 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.847996 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:23.848126 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:23.848189 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.848308 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.848329 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:40:23.851958 30040 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:23.851105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44880 (local address 127.29.142.188:46141)
0806 06:40:23.851149 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.851161 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.851353 (+ 192us) server_negotiation.cc:407] Connection header received
0806 06:40:23.851463 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.851464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.851483 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.851513 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.851904 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44880: BlockingRecv error: recv got EOF from 127.0.0.1:44880 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:40:23.857618 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.854258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35544 (local address 127.29.142.189:34327)
0806 06:40:23.854297 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.854299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.854541 (+ 242us) server_negotiation.cc:407] Connection header received
0806 06:40:23.854635 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.854636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.854645 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.854708 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:23.854885 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.855652 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.856762 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.857060 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.857101 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.857351 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:23.857388 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:23.857414 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.857512 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.857537 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:40:23.861030 30040 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:23.860240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.29.142.188:46141)
0806 06:40:23.860281 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.860283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.860469 (+ 186us) server_negotiation.cc:407] Connection header received
0806 06:40:23.860567 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.860568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.860580 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.860609 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.860974 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44884: BlockingRecv error: recv got EOF from 127.0.0.1:44884 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:40:23.870420 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.867042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35546 (local address 127.29.142.189:34327)
0806 06:40:23.867085 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.867087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.867376 (+ 289us) server_negotiation.cc:407] Connection header received
0806 06:40:23.867480 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.867481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.867491 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.867521 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:23.867706 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.868476 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.869611 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.869906 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.869946 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.870138 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:23.870182 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:23.870210 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.870317 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.870336 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:40:23.873772 30040 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:23.872954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44892 (local address 127.29.142.188:46141)
0806 06:40:23.872997 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.872999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.873194 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:40:23.873295 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.873296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.873308 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.873339 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.873711 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44892: BlockingRecv error: recv got EOF from 127.0.0.1:44892 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:40:23.882761 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.879168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35562 (local address 127.29.142.189:34327)
0806 06:40:23.879244 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.879246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.879519 (+ 273us) server_negotiation.cc:407] Connection header received
0806 06:40:23.879625 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.879627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.879638 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.879668 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:23.879854 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.880682 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.881879 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.882183 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.882226 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.882425 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:23.882458 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:23.882487 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.882592 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.882611 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 06:40:23.886114 30040 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:23.885307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.29.142.188:46141)
0806 06:40:23.885351 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.885353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.885538 (+ 185us) server_negotiation.cc:407] Connection header received
0806 06:40:23.885636 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.885637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.885648 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.885680 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.886052 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44900: BlockingRecv error: recv got EOF from 127.0.0.1:44900 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:40:23.898981 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.895563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.29.142.189:34327)
0806 06:40:23.895607 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.895609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.895873 (+ 264us) server_negotiation.cc:407] Connection header received
0806 06:40:23.895976 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.895977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.895987 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.896017 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:23.896216 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.896925 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.898078 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.898384 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.898425 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.898631 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:23.898733 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:23.898764 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.898874 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.898894 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:40:23.902464 30040 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:23.901554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44916 (local address 127.29.142.188:46141)
0806 06:40:23.901600 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.901602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.901793 (+ 191us) server_negotiation.cc:407] Connection header received
0806 06:40:23.901892 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.901893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.901905 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.901935 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.902406 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44916: BlockingRecv error: recv got EOF from 127.0.0.1:44916 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:40:23.923586 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.920007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35578 (local address 127.29.142.189:34327)
0806 06:40:23.920055 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.920057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.920395 (+ 338us) server_negotiation.cc:407] Connection header received
0806 06:40:23.920513 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.920514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.920527 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.920559 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:23.920770 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.921528 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.922724 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.923011 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.923064 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.923302 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:23.923337 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:23.923367 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.923479 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.923501 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 06:40:23.927381 30040 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:23.926338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44924 (local address 127.29.142.188:46141)
0806 06:40:23.926383 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.926385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.926644 (+ 259us) server_negotiation.cc:407] Connection header received
0806 06:40:23.926833 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.926834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.926847 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.926879 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.927324 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44924: BlockingRecv error: recv got EOF from 127.0.0.1:44924 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:40:23.963827 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:23.960965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35588 (local address 127.29.142.189:34327)
0806 06:40:23.961024 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.961026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.961145 (+ 119us) server_negotiation.cc:407] Connection header received
0806 06:40:23.961190 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.961191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.961203 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.961281 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:23.961432 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.962181 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.962966 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:23.963320 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:23.963398 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:23.963522 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:23.963585 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:23.963639 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:23.963722 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 06:40:23.963744 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 06:40:23.966574 30040 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:23.965922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44936 (local address 127.29.142.188:46141)
0806 06:40:23.965998 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:23.966000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:23.966010 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:40:23.966081 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:23.966082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:23.966092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:23.966157 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:23.966502 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44936: BlockingRecv error: recv got EOF from 127.0.0.1:44936 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:40:24.034721 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:24.031906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35594 (local address 127.29.142.189:34327)
0806 06:40:24.031975 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:24.031978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:24.032087 (+ 109us) server_negotiation.cc:407] Connection header received
0806 06:40:24.032125 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:24.032126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:24.032137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:24.032212 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:24.032340 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.033042 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.033876 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.034169 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.034247 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:24.034412 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:24.034452 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:24.034499 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:24.034558 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 06:40:24.034576 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 06:40:24.037426 30040 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:24.036703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44950 (local address 127.29.142.188:46141)
0806 06:40:24.036775 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:24.036778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:24.036874 (+ 96us) server_negotiation.cc:407] Connection header received
0806 06:40:24.036946 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:24.036947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:24.036961 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:24.037007 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:24.037364 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44950: BlockingRecv error: recv got EOF from 127.0.0.1:44950 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:40:24.169447 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:24.166824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35606 (local address 127.29.142.189:34327)
0806 06:40:24.166918 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:24.166920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:24.167029 (+ 109us) server_negotiation.cc:407] Connection header received
0806 06:40:24.167115 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:24.167116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:24.167129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:24.167206 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:24.167361 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.167868 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.168707 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.168914 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.168998 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:24.169158 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:24.169197 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:24.169262 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:24.169326 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 06:40:24.169347 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 06:40:24.172291 30040 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:24.171507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.29.142.188:46141)
0806 06:40:24.171599 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:24.171601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:24.171612 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:40:24.171757 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:24.171758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:24.171772 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:24.171840 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:24.172223 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44966: BlockingRecv error: recv got EOF from 127.0.0.1:44966 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 06:40:24.303120 30089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3afb55b90f294c0c82e59798422b06ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3afb55b90f294c0c82e59798422b06ea" member_type: VOTER last_known_addr { host: "127.29.142.190" port: 36863 } } peers { permanent_uuid: "2ae7491358ff45ffb9692d43678b05e3" member_type: VOTER last_known_addr { host: "127.29.142.189" port: 34327 } } peers { permanent_uuid: "15c304e120944f8f874f0197dca0522b" member_type: VOTER last_known_addr { host: "127.29.142.188" port: 46141 } } }
I20260806 06:40:24.303359 30089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3afb55b90f294c0c82e59798422b06ea [sys.catalog]: This master's current role is: LEADER
I20260806 06:40:24.431778 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:24.429596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35608 (local address 127.29.142.189:34327)
0806 06:40:24.429665 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:24.429666 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:24.429702 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:40:24.429730 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:24.429731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:24.429741 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:24.429807 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:24.429942 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.430428 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.431196 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.431379 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.431435 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:24.431560 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:24.431587 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:24.431625 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:24.431696 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 06:40:24.431713 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 06:40:24.434425 30040 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:40:24.433594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44968 (local address 127.29.142.188:46141)
0806 06:40:24.433764 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:24.433765 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:24.433781 (+ 16us) server_negotiation.cc:407] Connection header received
0806 06:40:24.433804 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:24.433805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:24.433815 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:24.433866 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:40:24.434358 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44968: BlockingRecv error: recv got EOF from 127.0.0.1:44968 (error 108)
Metrics: {"server-negotiator.queue_time_us":143}
I20260806 06:40:24.737772 30057 catalog_manager.cc:1259] Loaded cluster ID: 7e4c7a3905224ccf98c9b75bb59adf8d
I20260806 06:40:24.737879 30057 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b: loading cluster ID for follower catalog manager: success
I20260806 06:40:24.738788 30057 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b: acquiring CA information for follower catalog manager: success
I20260806 06:40:24.739199 30057 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 15c304e120944f8f874f0197dca0522b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:40:24.950133 30307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:24.947751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35610 (local address 127.29.142.189:34327)
0806 06:40:24.947882 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:24.947884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:24.947934 (+ 50us) server_negotiation.cc:407] Connection header received
0806 06:40:24.947997 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:24.947997 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:24.948009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:24.948074 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:24.948208 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.948713 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.949495 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.949684 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.949747 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:24.949877 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:24.949905 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:24.949944 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:24.949996 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:40:24.950015 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260806 06:40:24.955044 30314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:24.952108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44982 (local address 127.29.142.188:46141)
0806 06:40:24.952339 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:24.952341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:24.952351 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:40:24.952384 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:24.952385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:24.952439 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:24.952516 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:24.952632 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.953112 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.953980 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.954224 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.954299 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:24.954477 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:24.954621 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:24.954898 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:24.954954 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 06:40:24.954974 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":185,"threads_started":1}
I20260806 06:40:24.959867 30314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:24.957076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44990 (local address 127.29.142.188:46141)
0806 06:40:24.957130 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:24.957132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:24.957146 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:40:24.957172 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:24.957173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:24.957186 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:24.957240 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:24.957384 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.958208 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.959146 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.959343 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.959416 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:24.959567 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:24.959605 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:24.959667 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:24.959727 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 06:40:24.959751 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:40:24.959968 30307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:24.956810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35614 (local address 127.29.142.189:34327)
0806 06:40:24.956857 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:24.956860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:24.956944 (+ 84us) server_negotiation.cc:407] Connection header received
0806 06:40:24.957044 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:24.957045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:24.957058 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:24.957109 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:24.957282 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.957877 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.958881 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.959239 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.959316 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:24.959523 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:24.959562 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:24.959612 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:24.959854 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 06:40:24.959882 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:40:24.960742 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:40:24.956825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47000 (local address 127.29.142.190:36863)
0806 06:40:24.956939 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 06:40:24.956942 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:40:24.957093 (+ 151us) server_negotiation.cc:407] Connection header received
0806 06:40:24.957261 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:40:24.957263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:40:24.957324 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:40:24.957430 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:40:24.957504 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.958341 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.959478 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:40:24.959791 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:40:24.959861 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:40:24.960270 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:40:24.960465 (+ 195us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:40:24.960498 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0806 06:40:24.960636 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 06:40:24.960667 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":68,"threads_started":1}
I20260806 06:40:24.962628 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 29725
I20260806 06:40:24.975307 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 29828
I20260806 06:40:24.988667 30266 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrotugL/build/release/bin/kudu with pid 29935
2026-08-06T06:40:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3090 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6171 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (287230 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST